Event Processing

The data rate supplied by the first processing stage is still too high to allow persistent storage. Algorithms coded in hardware, even more sophisticated pattern recognition or exploitation of digital signal processors (DSP) do not satisfy an important requirement - flexibility. Although specialised event identification farms have been built using DSPs[ea97], limited resources of the processing boards make it difficult to provide ease of programming. Therefore specialised processing capability is sacrificed for having a general purpose processing environment. The overhead of the operating system and the lower performance of off-the-shelf desktop computers are compensated by multiplying the size of the parallel processing system. This solution is preferable to farms of specialised processing boards, especially for large systems like the CMS detector . Data from subdetectors are not requested from front end crates directly, but from a large quantity of units that are capable of buffering a certain number of event fragments. These buffers, thereafter called Readout Units (see pzd172 in figure 14) must provide enough intelligence to serve requests for specific data portions and report erroneous behaviour. They have to manage large memory banks locally. The $n$ to $n$ relationship of data holders and data processors, where $n$ is in the order of 500, requires management intelligence that can only be provided by general purpose computer systems. As depicted in the left part of figure 9 the processing elements, called Filter Units (see also pzd172 in figure 14) have to build the event structure by requesting single event pieces from the readout units. The reader that is familiar with the design of tightly coupled multiprocessors[Dun90] will immediately recognise the similarities (outlined in the right part of the figure). As in a shared memory multiprocessor, the filter units access ``their'' memories using a switching fabric. Processing of the data is done in the local cache of a processing node. This refers to applying event filter algorithms to the data fragments that have been retrieved from the readout units, or ``shared memories''. In fact the similarities go even deeper, as the problem that has to be solved can be mapped to a MIMD architecture[Fly66]. The filter units operate on disjoint data sets. Each node processes one event independently of the others. Unlike a tightly coupled multiprocessor system, the entire set of readout units does not form an address space that can be seen by all filter units. As the readout units are processors with local memories themselves, messages have to be passed between readout unit and filter unit clusters. Still, the aim is to provide the abstraction of a single address space spanning the memories of the readout units for each filter unit (see figure 10). The data in the readout units represent a time series of measurement values. As each event is however processed by exactly one filter unit, the time-contiguous memory abstraction need not be shared between the nodes of the filter farm. In this respect, the system differs substantially from a distributed shared memory system[Li86,LH89][CDK94, pp. 518].

Although the data rate is reduced by rejecting events, not all data can be provided to the filter units. At an input rate of 100 GBytes/sec, each of the foreseen 500 readout units would feed data at $1.6$ Gb/sec into the network that connects memories and processors. These $1.6$ Gb/sec contain $100 \times 10^3$ events. This means that one readout unit sends $1.6 \times 10^9 {\rm bps} / 500 {\rm destinations} = 3.2 {\rm Mbps}$ to each destination. Each destination gets this rate from 500 sources as input which results again in 1.6 Gbps total input rate per filter unit. A non blocking multistage switching fabric with a nominal capacity of 2 Gb/sec should be able to handle this load. However, this assumption should not be made for two reasons: