VME in CMS

Printable (html)
Printable  (pdf)
1 Introduction
2 VME64x in CMS
3 VMEBridges
    3.1 CAEN
        3.1.1 Technical Info
        3.1.2 Status
        3.1.3 Software
        3.1.4 Firmware
        3.1.5 Measurements
    3.2 SBS

Performance Measurements

The VME Hardware was a optical VME bridge from CAEN (V2718_Kit) and a RAMix139 VME memory (32Mb). The PC was a PIII 800 MHz PC running Linux with a 2.4.x kernel. The measurements have been carried out with
  • CAEN library libCAENVME.so.2.1
  • firmware for V2718 : v1718vub_rev0.11.rbf
  • firmware for A2818 : a2818pcb_rev0.5.rbf
  • firmware for A2719 : a2719ci_rev0.3.rbf
All measurements used the HAL library(version ver-03-05-test-04).

A measurement with the usb version of the CAEN Bridge (Model V2718_Kit) cane be looked at here. (The measurement has been obtained with the firmware v1718vub_rev0.12.rbf for the V1718 board. The rest of the measurement conditions have been identical to those for the optical bridge.)

Performance for single accesses

This test was made running the "PerformanceTester" application contained in the HAL examples. It has been run one time with the dummy BusAdapter and one time with the BusAdapter for the VME Bridge. This allows to subtract the overhead due to the HAL library itself.

Results using the VMEDummyBusAdapter


===============================================================================
ATTENTION ==> the measured time values do not measure the process time but
              the total time elapsed during the measurement (laboratory-time)
              So, in order to get meaningfull measurements,  be sure not to  
              have other active processes on your machine.
===============================================================================


(TimeoutException) Timeout during polling the item "rwFlagMask"
      1000ms have passed wheras the timeout has been set to 1000ms
      3680000 polls have been carried out
      (HardwareDevice::pollItem)

read                      r/w-item with full mask : 2.57833us
read           r/w-item with full mask and offset : 2.95839us
write                     r/w-item with full mask : 2.99179us
write          r/w-item with full mask and offset : 3.42508us
read                        r-item with full mask : 2.53569us
read             r-item with full mask and offset : 2.91862us
write                       w-item with full mask : 2.66218us
write             w-item with full mask and offset: 3.07601us
unmaskedRead              r/w-item with full mask : 2.66056us
unmaskedRead   r/w-item with full mask and offset : 2.90977us
unmaskedWrite             r/w-item with full mask : 2.65765us
unmaskedWrite  r/w-item with full mask and offset : 2.92356us
setBit                    r/w-item with 1bit mask : 3.26963us
setBit         r/w-item with 1bit mask and offset : 3.71627us
resetBit                  r/w-item with 1bit mask : 3.26833us
resetBit       r/w-item with 1bit mask and offset : 3.69591us
setBit                      w-item with 1bit mask : 2.95803us
setBit           w-item with 1bit mask and offset : 3.35657us
resetBit                    w-item with 1bit mask : 2.85544us
resetBit         w-item with 1bit mask and offset : 3.37863us
===============================================================================

Results using the CAEN2718LinuxPCIBusAdapter

Remark: The items which need significant more time involve two VME accesses. (This is documented in the HAL documentation). In order to obtain the overhead due to the driver and the hardware the measured values of the values above measured with the DummyBusadapter must be subtracted from the values below.

===============================================================================
ATTENTION ==> the measured time values do not measure the process time but
              the total time elapsed during the measurement (laboratory-time)
              So, in order to get meaningfull measurements,  be sure not to  
              have other active processes on your machine.
===============================================================================


(TimeoutException) Timeout during polling the item "rwFlagMask"
      1009ms have passed wheras the timeout has been set to 1000ms
      100000 polls have been carried out
      (HardwareDevice::pollItem)

read                      r/w-item with full mask : 14.2928us
read           r/w-item with full mask and offset : 14.7548us
write                     r/w-item with full mask : 24.1194us
write          r/w-item with full mask and offset : 24.6097us
read                        r-item with full mask : 14.1733us
read             r-item with full mask and offset : 14.7672us
write                       w-item with full mask : 13.5404us
write             w-item with full mask and offset: 14.0083us
unmaskedRead              r/w-item with full mask : 14.1884us
unmaskedRead   r/w-item with full mask and offset : 14.8598us
unmaskedWrite             r/w-item with full mask : 13.7226us
unmaskedWrite  r/w-item with full mask and offset : 14.3353us
setBit                    r/w-item with 1bit mask : 24.3967us
setBit         r/w-item with 1bit mask and offset : 24.9829us
resetBit                  r/w-item with 1bit mask : 24.3639us
resetBit       r/w-item with 1bit mask and offset : 24.9634us
setBit                      w-item with 1bit mask : 14.0308us
setBit           w-item with 1bit mask and offset : 14.5417us
resetBit                    w-item with 1bit mask : 13.821us
resetBit         w-item with 1bit mask and offset : 14.4us
===============================================================================

Performance for Block Transfers

Measurement conditions:
  • The measurements exclusively measure the block transfer performance of the VME bridge.
  • Data blocks have been transferred for different sizes from the host via the VME Bridge to the memory module and vice versa. The data has been generated with a random generator.
  • The size of the data blocks has been varied from 256 bytes to 32Mb. Altogether 12 blocksizes where chosen. The data read back was always verified against the data previously written. No data error has been observed.
  • The measurement at each data point was repeated 100 times. The mean values of the resulting transfer rates are summarized in the colored plot below.
  • Additional plots have been produced, in order to show the distribution of the data throughput for the 100 measurements taken for at each block-size.

Results

Read and write performance as a function of the block size.

Read / Write Block Transfer of optical CAEN bridge with HAL


Distribution of read throughput for fixed block sizes (100 Measurements).

Read Block Transfer performance distribution for fixed blocksize of optical CAEN bridge with HAL
Read Block Transfer performance distribution for fixed blocksize of optical CAEN bridge with HAL


Distribution of write throughput for fixed block sizes (100 Measurements).

Write Block Transfer performance distribution for fixed blocksize of optical CAEN bridge with HAL
Write Block Transfer performance distribution for fixed blocksize of optical CAEN bridge with HAL

last changed: 2012-10-02 comments to: Christoph Schwick