Speed

Speed is a key requirement for data acquisition. The amount of data that is recorded, processed and stored has to be maximised under a given configuration. As communication technology is advancing quickly the gap between hardware and software is growing. We have to address this problem if we want to achieve our goal. When talking about speed we distinguish between two properties: Throughput and Latency.

Throughput stands for the number of payload bytes that can be transferred per second. Latency is the time to transfer a single message with no payload. Satisfying the requirements of high throughput and low latency is difficult, as the two qualities are contradictory: message processing overhead (including the protocol) is a limiting factor to performance. Therefore one aims at decreasing the ratio of processing to transfer time. This can be achieved by increasing package size. This approach is known as scatter/gathering[GJO00]. Figure 11 depicts the described effect. Large packages however are not suitable for sending commands at high frequencies. As can be seen in the leftmost part of the slope in figure 11, throughput decreases with smaller payload sizes. So, less data can be transferred in the same amount of time with smaller packages, than with larger ones. The dilemma spreads to higher protocol layers and favours the end-to-end argument [SRC84]: Transferring a file requires different means than sending small control datagrams. For the former, the overhead of connection establishment, a sliding window protocol with acknowledgement of multiple packages and large packages is accepted for gaining throughput. The same approach for a massive parallel system that sends around single numbers would result in low efficiency. Messages are small and per packet overhead as to be kept small.

In data acquisition systems for high energy physics we encounter both communication patterns: sending data requests, input buffer flushing commands, and in the case of distributed event building remote function calls with return values, represent small messages that have to be transferred within a short amount of time. Event fragments can still be counted as small messages when applied to Gigabit network technology. These small messages are issued at very high rates which means that high throughput is also required. The following requirement can be derived:

Requirement 1 (Speed)   Communication software in high performance distributed systems shall make good use of the underlying interconnection technology for every kind of observed communication pattern.

Rationale: This means that the communication subsystem must deliver as much of the available network performance to the application as possible. To meet the application's requirement an appropriate communication means has to be identified. As a negative example one could mention selecting TCP as a transport protocol for highly parallel computing in clusters of workstations.