The Receiver Architecture

A possible lean server architecture is outlined in C++ pseudocode in figures 35 and 36. According to the trace diagram in figure 33 a thread ( RequestHandler in the code example), which may be borrowed from the main program, receives messages from the communication link. At this point, a decision has to be taken concerning the used concurrency mechanism. Either the message is immediately handled by a dispatcher, or it is forwarded to another thread ( theInvocationTask in the code example) using a queue for each task. This InvocationTask gets the message from its internal queue. First it cuts off the operation identifier. Then it obtains a pointer to the method that has to be called. A pointer to the parameter list is passed with the activation of the servant skeleton. The skeleton must interpret this byte stream correctly and forward the parameters to the servant implementation.

This model corresponds to a worker pool architecture. The solution in which the thread that receives packages from the network directly makes the call to the servant can be either used for a single threaded approach or for a leader-follower[Sch98] architecture. We will outline this option in section 5.6. It is also possible to have only one queue between receiver and the workers, but this degrades possible parallelism as more than two threads will compete for the shared resource. We experienced that multithreaded servers must be chosen carefully in high performance systems, as locking and thread switching can decrease performance. Therefore a multithreaded architecture is favourable over reactive handling if $t_{Exec} > t_{Sync} + t_{Cswitch}$, where $t_{Exec}$ is the run time of the invoked servant method, $t_{Sync}$ is the time needed for performing synchronisation primitives associated with passing the message to the destination thread. $t_{Cswitch}$ is the platform context switching overhead. These last two values can only be interleaved with the servant execution and are not present in reactive mode. Therefore they have to be masked by the application programmer supplied code in order to improve on the synchronous processing. The protocol used for encoding objects and methods onto the message need not be standard in system like ours. We argue that in high performance distributed systems it is quite justifiable to use non-standard approaches in order to meet the requirements. Upper layers however should follow a unified interface. Such loose coupling will allow the lower transport architecture to be transparently exchanged.

For this level we suggest a data transfer component that relies on a minimal protocol as it can be seen in figure 34. The length of the incoming message constitutes an essential piece of information for every communication between two processes: knowing how much data have to be received will allow the building of efficient message parsers. The length may include or exclude the size of the header; it may be given in bytes, words or another unit. In any case it allows the evasion of subsequent read operations in order to get the full message on the lowest level. Next, an association between a sender and a receiver is needed. It may simulate connection oriented operation if the underlying network services do not provide such functionality. But it may also serve for other purposes: identification of the message originator for logging, debugging, security and most important, efficient demultiplexing. A channel identifier consists not only of the originator, but also of the destination identification. Thus the receiver can use it for figuring out if at all it needs to process this message. This can be useful for implementing one-to-many communication. How is it now possible to use such a virtual channel identifier (VCI) for efficient demultiplexing of a message? Suppose we use ordinary programming interfaces such as sockets or TLI for interfacing to the network. Then we would have to block on a select system call in Unix like environments in order to determine, whether a file descriptor, i.e. a connection has data that are ready to be received. The select call does not tell which connection is ready. This has to be determined by looping over all established connections. An alternative is to accept each connection in a separate thread and block on a read operating system call that takes the file descriptor associated with the connection as an argument. Both approaches do not scale well when confronted by several hundreds or thousands of clients. Therefore, we wish to apply a connectionless protocol for environments like ours, in which we have to cope with the presence of many originators. In the case of high level usage with operating system calls such as select, we receive the message and extract the VCI information. This identifier can be used as a key in a map for finding associated information on what to do with messages from this source. As the VCI can easily be split into receiver and sender parts (e.g. 16 bit for the receiver and 16 bit for the sender in a 32 bit VCI field), selective operations with low latencies can be implemented. In case of low level device access the VCI can even be extracted at this low level. When matching it with a configuration table the message can be demultiplexed to a process or thread at driver level directly. This early demultiplexing scheme provides further performance improvement of communication software[KSLB99].

Then, an identifier for the message itself is needed. There are several ways to implement this. One possibility is to increase a counter or a timestamp. Another possibility to efficiently use this field is the magic-cookie pattern[GJO00], whereby with every invocation the message identifier is supplied by the client. The server has to make a response in which the cookie is returned to the client. This technique can be used to build reliable protocols over unreliable channels. Duplicate or out of order messages can be detected.

The call identifier is the information that contains the purpose of the message. It allows the receiver to dispatch the invocation to the requested method. An example is to encode the $id_{Call} = (id_{Object}, id_{Method})$ tuple into an integer and to add the parameters and parameter meta information directly after this identifier. Combination of object and method identifier allows to efficiently demultiplex the invocation request in one step. It is no longer necessary to find the object first and later resolve the pointer to the method with the help of data structures that are local to the object. This approach can be found in first generation ORBs[GS96,CHY$^$98]. Instead a common data structure holding information about all objects and methods is used. This does not imply that the servant class has to be instantiated already before invocation. Instantiation can also be done on demand, as the invocation tuple identifies a servant class, not a specific servant instance. Still the problem persists of how to identify different instantiations of a servant class. All this can be encoded into the single invocation tuple as well. We do not want to present the complete architecture as it has been implemented (see also appendix A.2). We simply shed light on possible design choices and feasible ways of implementations. One of several choices is to split up the object identifier into several regions: Using a 32 bit integer, 8 bit could be used for encoding the object type, 8 bit for the method in the object and the remaining 16 bit for the instance number. If the instance is set to zero, it is the object adapter´s task to create a new one and assign a valid identifier to it. 8 bit for encoding the object type gives us 256 possible servant classes per process. This is a reasonable number. Also, only a few classes have more than 256 methods. However it may be important to manage a large amount of concrete objects. This is the case in physics applications where we have to deal with a large number of data objects (e.g. crystals in a detector, electrons in a space-time volume). Clearly there is no limit on sliding these boundaries or using other, maybe larger data types for encoding. The essential approach however is to encode these three items, servant id, method id, instance id into one atomic entity for efficient processing.

Concerning the upcall parameters, it is the skeleton´s responsibility to extract the information and forward the parameters to the implementation. Doing so allows precompiled demarshaling routines. Their advantage over an interpreted approach in the ORB core is faster ``digestion'' of this information. As a disadvantage we encounter increased code size due to the static method encoding/decoding routines. There are only two cases that have to be distinguished when encoding data types into the payload: (i) single entities and (ii) sequences. It is not necessary to encode the number of parameters into the message. As the skeletons pick the data from the stream they know how many parameters they have to process. So for single values the data can be streamified on the client side and put into the message directly; for sequences it is necessary to add the number of elements in the sequence just before the first element of the list. Depending on the environments´ needs, the parameters may be encoded into some machine independent format. This task can be performed in the stubs and skeletons in order to keep the ORB core small and efficient. The $id_{Call}$ can be looked up in an automatically generated object table; how the table is implemented need not to be specified. If there is no need for storing several instances of a servant class, but there is only one object per class, then an ordinary array is sufficient. This table enforces static binding of the clients to the servers. In other cases when dynamic modification of the object/method table is needed, a map or a hashtable can serve for storing the (object, method, instance) triplets efficiently. A precompiler may generate automatically consecutive numbers for the tuples, representing table keys for the operations defined in IDL. On object instantiation the stubs and skeletons produced from the IDL definitions store the pointer to the operation in the table using the operation indices. Retrieval is just a matter of accessing the array with the previously generated $id_{Call}$. Such $O(1)$ algorithms are scalable. It is however not possible to add more classes to the server dynamically, because the identifier for each type is determined during the precompilation phase. Skeletons inherit from a common base class (see example 37). They have an entry point for each operation which is called from the InvocationTask service routine. A pointer to the location within the received message at which the call parameters start is passed. These have to be decoded and passed to the concrete server implementation. Instances of the servant are held by each skeleton. Doing so allows efficient implementation without using inheritance or virtual function calls.