next up previous contents
Next: An Assessment of CORBA Up: Computing Clusters Previous: ASCI RED   Contents

WEB Search Engines

A completely different application area, namely WEB search engines have been addressed in a cluster architecture that has been commercialised by the Inktomi corporation. Regarding the architectural overview[FGC$^$97] in figure 19 and comparing it with the architecture of the CMS data acquisition system (see figure 14) that dates back to 1995, we can immediately recognise the similarity: both architectures aim at a symmetric design, splitting the system into a data server and a data processing part. Both parts are $n$ to $n$ interconnected. The Inktomi cluster uses Myrinet, a 1 Gbps wormwhole-routing based interconnect with intelligent network interface cards as intermachine communication medium. The system is not a massive parallel one. A request for information that has been issued from a WEB browser is processed on one front end workstation. The request however will make it necessary to collect a set of diverse data and these are retrieved from several nodes in the worker pool. Once the results are available they are sent to the responsible front end workstation that performs the translation into HTML code and serves the result to the requesting WEB browser. The architecture is also deployed in the HotBot search engine. Both projects have their roots in the 1997 ended Berkeley Network of Workstations (NOW) project[ACP95].


\begin{Figure}
% latex2html id marker 770\begin{center}
\epsfig {figure=inkt...
...i
WEB search engine. Picture taken from \cite{Fox97}.} \end{center}\end{Figure}

We could not find out about the current system size of the Inktomi cluster. The HotBot search engine which uses the same architecture, comprised 60 nodes in 1997[FGC$^$97]. The actual system size is however not the most important issue. Rather than that, it is interesting to see, that distributed data acquisition is much more related to the problems that we encounter in the Internet domain than in the parallel computing community. This lets us conclude, that research in the area of distributed data acquisition systems does have significant spin-off capacities to industry applications.

From the work on other experiments' data acquisition systems and cluster computing initiatives we have seen that commercial off-the-shelf products are more and more common in our environment. Standard middleware for distributed object-oriented components is also designed by the Object Management Group. The architecture which is proposed by this consortium is called CORBA. It is widely used in industry and we are also interested in using it. Guaranteed support over the years and interoperability between a variety of value adding components are strong arguments for such choice. The following chapter gives an overview of this model and tests its usefulness with quantitative measurements of a state-of-the-art CORBA implementation.


next up previous contents
Next: An Assessment of CORBA Up: Computing Clusters Previous: ASCI RED   Contents
Johannes Gutleber
1999-10-29