Multi-client LAN/WAN Performance Analysis of Ninf:
a High-Performance Global Computing System

Abstract:

Rapid increase in speed and availability of network of supercomputers is making high-performance global computing possible, including our Ninf system. However, critical issues regarding system performance characteristics in global computing have been little investigated, especially under multi-client, multi-site WAN settings. In order to investigate the feasibility of Ninf and similar systems, we conducted benchmarks under various LAN and WAN environments, and observed the following results: 1) Given sufficient communication bandwidth, Ninf performance quickly overtakes client local performance, 2) current supercomputers are sufficient platforms for supporting Ninf and similar systems in terms of performance and OS fault resiliency, 3) for a vector-parallel machine (Cray J90), employing optimized data-parallel library is a better choice compared to conventional task-parallel execution employed for non-numerical data servers, 4) computationally intensive tasks such as EP can readily be supported under the current Ninf infrastructure, and 5) for communication-intensive applications such as Linpack, server CPU utilization dominates LAN performance, while communication bandwidth dominates WAN performance, and furthermore, aggregate bandwidth could be sustained for multiple clients located at different Internet sites; as a result, distribution of multiple tasks to computing servers on different networks would be essential for achieving higher client-observed performance. Our results are not necessarily restricted to the Ninf system, but rather, would be applicable to other similar global computing systems.

Keywords:

 Global network computing, Performance evaluation

1 Introduction

Rapid increase in speed and availability of network of supercomputers is now making so-called high-performance `global computing' possible, in which computational and data resources in the network are collectively employed to solve large-scale problems. There have been several recent proposals of global computing infrastructures, including our Ninf (Network Infrastructure for global computing) [1, 2] system, which makes available multiple remote computing and database servers on the network, allowing clients to semi-transparently call remote numerical applications and libraries from languages such as Fortran, C, and Java. Multiple calls in a network are coordinated by metaservers, which keep track of server load/availability, network bandwidth, etc. WWW-based interfaces are also available for easy user access of remote computing and data.

On the other hand, we have so far found that much of system performance characteristics and criteria for enabling the feasibility global computing have not been well-investigated. In order for global computing to succeed, we need to investigate and clarify various characteristics of the technology that are main constituents of a global computing system:

Communication Performance:

Global computing requires large amounts of data to be shipped over the network. Although available bandwidth is rapidly increasing, no quantitative measures are provided to judge its sufficiency. Also, since latency will not improve drastically in the future, it is not clear whether high latency would not be a deterrent to attaining performance.

Computing Server Selection and Performance:

Computation will be typically performed on high-performance supercomputing servers such as vector supercomputers and MPPs. On the other hand, they will be shared amongst multiple remote clients from different places in the global network. It is not clear what criteria should be employed in selecting such supercomputing servers, and whether the current supercomputers/MPPs would be appropriate for such usage.

Sharing by Multiple Clients:

Computing servers will be shared by arbitrary multiple clients on the network. Thus, it must also be shown that the combination of the Ninf computing server (and those in similar systems), supercomputers, and their operating systems, could handle the concentration of requests from multiple clients in a graceful manner, under both LAN and WAN environments. Since we rely on existing supercomputers, it is not apparent whether their operating systems could handle multiple computational requests from clients, whose transactions will tend to be longer duration compared to typical short database query transactions. Also, it must be shown that the system is resilient to various faults that could occur in network computing.

Remote Library Design and Reuse:

When executing remote libraries registered on Ninf computing servers, there is an option as to whether to distribute the computing resources amongst different client requests in a task parallel manner, or to allocate all the processors to each client task in a data parallel manner in sequence. This directly influences the design of the libraries, whether the client library has to be prepared specifically for Ninf, or the best (parallel) library for the machine could be directly reused and utilized.

We report on the benchmark results of Ninf computing server performance under various LAN and WAN environments to investigate the feasibility of global computing. As a representative of typical application core, we have selected the Linpack benchmark which involves significant amount of data transfer compared to its problem size, and NASPar EP benchmark application, in which communication is minimal. We have conducted the benchmarks in LAN and WAN environments with various combinations of client and server machines. We have also tested and compared multi-client cases in which the clients are located in the same LAN, on multiple nodes in the same remote network, and multiple clients located on a different network. As a result, we have observed the following results:

• We have observed that, with higher network bandwidth, Ninf execution becomes faster than local execution with a relatively small problem size for Linpack and obviously for EP. Here, latency was not a significant issue due to larger grain size of the problem; thus, given sufficient WAN bandwidth, global computing in the wide-area will be quite feasible in the future for a wide class of computational-intensive problems.
• By employing parallel vector machines (Cray J-90, 4PE) and large SMPs (Sun SPARC SMP), we have confirmed that operating systems for such machines are sufficient both functionally and performance-wise for supporting multiple clients over both WAN and LAN. Thus, we claim that, Ninf (and other global computing platforms) could readily be built on existing platforms.
• For J90, data-parallel execution using optimized libraries would be faster for lightly-loaded cases, and would exhibit almost identical performance to task-parallel execution under heavily-loaded cases. Thus, using optimizing libraries maximizing the performance of parallel machines would be appropriate under both LAN and WAN situations. On the other hand, for SMPs, preliminary results show that highly-multithreaded libraries exhibit slowdown in performance in multi-client situations due to thread-switching overhead, and there is a need for determining the optimal number of threads versus the number of clients.
• For compute intensive applications such as EP with little communication, LAN vs. WAN performance would be essentially equivalent, achieving high computing server utilization. Such applications are numerous, such as parallel rendering/imaging, and parameter sensitivity analysis, and they could readily benefit from Ninf and other global computing systems.
• For Linpack, which requires extensive communication, LAN performance was acceptable for practically-sized data sets. For WAN execution, performance will heavily depend on aggregate network throughput of the client, and not on server load, which remained light. Thus, under communication-dominant cases, task assignment and distribution should not be merely based on server load and utilization information, but rather on achievable network bandwidth.

2 Overview of Ninf System

The Ninf system is currently based on the client-server model. The client can make use of various computing library and database resources via server processes on supercomputers and database servers. Figure 1 illustrates the computational aspect of the Ninf system. Ninf users can easily utilize the supercomputing power from his desktop client over WAN without steep learning curves or major modifications to his applications.

The Ninf system consists of Ninf computational and database servers, Ninf client API which defines the client bindings, and Ninf metaservers which perform wide-area detection, allocation, and scheduling of resources over the network. The clients and various servers communicate via Ninf RPC (Remote Procedure Call) [3], which is tailored for the needs of high-performance numerical computing. There are also various supportive tools such as server registry tools, stub generators, query proxies, GUI builders, etc.

For the purposes of this paper, henceforth we will concentrate on describing the computational aspects of Ninf.

2.1 Ninf Computational Server

The Ninf computational server is a process which services remote computing requests of remote clients by managing the communication and activation of the services requested via Ninf RPC. Binaries of computing libraries and applications are registered on the server process as Ninf executables, which can be semi-automatically generated with IDL descriptions using the Ninf stub generator. The underlying transfer protocol is Sun XDR on TCP/IP, allowing easy porting on most major supercomputer platforms.

2.2 Ninf Client API

 Ninf Client API is defined for major programming languages such as Fortran, C, C++, and Java. Ninf_call is a representative API used for invoking a named remote library on the server as if it were on a local machine via Ninf RPC. For example, one might call a local matrix multiply library as:

double A[n][n], B[n][n], C[n][n];
dmmul(n, A, B, C);

while with Ninf, one invokes the remote library with:

Ninf_call("dmmul", n, A, B, C);
// or, Ninf_call("http://.../dmmul",...

There are also APIs for asynchronous RPCs (Ninf_call_async), specification of transactions for parallel execution (Ninf_transaction_begin, Ninf_transaction_end), and queries of numerical database (Ninf_query, etc.).

2.3 Ninf IDL

Ninf executables can be derived directly from existing numerical libraries such as LAPACK. Each subroutine in the library become callable via Ninf RPC by specifying their interface information with Ninf IDL (Interface Description Language). In the matrix multiply example, the IDL for the actual multiply routine on the server would be as follows:

Define dmmul(long mode_in int n,
mode_in double A[n][n],
mode_in double B[n][n],
mode_out double C[n][n])
"dmmul is double precision matrix multiply",
Required "libxxx.o"
Calls "C" mmul(n,A,B,C);

Argument numbers, types, access modes (input/output), array size, are necessary IDL information. As matrices are usually passed in memory with call-by-reference for local libraries whereas they must be shipped back and forth across the network for Ninf_call, matrix size, region of usage, stride, etc. that are dependent on scalar input arguments are either automatically inferred from IDL information or could be directly specified by the IDL writer. There are other optional info such as necessary modules for compilation and linking, aliases, client callback functions, comments on functionality, etc.

We also note that, with Ninf RPC, no prior downloading of IDL or stub generation is necessary on the client side. Stub generation are done solely on the server side; the client programmer never sees or manipulates the IDL information. As a result, Ninf_calls can be freely made without any library linking or header file inclusion. This is achieved by two-stage RPC in Ninf_call; when the client calls the server, it returns the compiled IDL information as interpretable code to the client. Ninf_call then interprets the IDL code and marshalls the arguments.¹

2.4 Ninf Metaserver

The Ninf metaserver monitors multiple Ninf computing servers on the network, and performs scheduling and load balancing of client requests. The client need not be aware (but could specify) the physical location of computing servers. Additionally, metaserver controls the parallel, fault-tolerant execution of multiple sequence of Ninf_calls. The block of code surrounded by Ninf_transaction_begin and Ninf_transaction_end are not executed immediately; rather, data-dependency graph of the Ninf_call arguments are dynamically created, and at the end of the code block, the metaserver schedules the computation to multiple computational servers accordingly.

3 Single Client Benchmarks

We first extensively benchmark Ninf single client performance on multiple client and server platforms. As a benchmark program, we employ Linpack and NASPar EP. The former requires extensive communication to ship dense matrices over the network, but becomes computation dominant for large matrices. The latter is computation dominant irrespective of its arguments, and is used for measuring the effectiveness of load balancing performed by the metaserver onto multiple Ninf servers.

3.1 Single Client Benchmarking Environment

For double precision Linpack, we execute the LU-decomposition (dgefa) and backward substitution (dgesl) remotely. The overall execution time of Ninf_call consists of communication time + computing time . Given a Linpack matrix size , they are:

where and are the setup times for communication and computation, respectively, is the client-server communication throughput, and is the local Linpack performance on the server machine. Then, Ninf_call performance is as follows:

Because is while is , it becomes computation dominant as becomes larger. As a result, we can expect that will exceed the performance on the client machine given sufficient .

The client and server machines in the benchmark are shown in Figure 2. We also show the client-server combination in Table. Local indicates the local Linpack performance without Ninf.

As Linpack routines, we registered the sgetrf and sgetrs of the libSci library, which make use of all 4 processors on the J90. On other machines, we employed glub4 and gslv4[5] routines which employ blocking optimizations and could thus be executed efficiently on RISC-based workstations (and thus would be disadvantageous for Ninf). Matrix size () was altered from 100 to 1600

3.2 Single Client Linpack Results

Figure 3 shows the results of SuperSPARC and UltraSPARC clients. The horizontal axis indicates the size of the matrix, and the vertical axis indicates the performance of Ninf_call and Local in Mflops. The performance of Local remains relatively constant across for both SPARCs. On the other hand, Ninf_call performance improves steadily as increases, and for both SPARCs, exhibited superior performance to Local at approximately .

Comparing UltraSPARC Local and SuperSPARC to UltraSPARC Ninf_call, the latter converges to UltraSPARC Local performance as becomes larger. This is because for large , hiding the communication overhead. The same is true for Alpha Local and Super/UltraSPARC to Alpha Ninf_call. For Ninf_calls to J90, J90's Local achieves 600Mflops when , while Ninf_call exhibited similar performance even for both SPARC clients whose Local performance differs considerably.

Figure 4 shows the results for Alpha clients. Here, we have additionally measured the performance of standard Linpack routines without blocking optimizations, in order to simulate the situations in which the users not taking the extra effort of optimizing his local library routine. Although recent performance improvements of PC and WS are significant, it is difficult for a user to write an optimized program for large-scale problems.

Comparing Alpha Local and Ninf_call to J90, Ninf_call become faster at approximately . On the other hand, when employing a standard, non-optimized routine on Alpha, Ninf_call became advantageous at approximately .

Figure 5 shows the communication throughput of Ninf_call. As Ninf sends data in XDR packets, marshalling/unmarshalling at both the client and the server, and communication inbetween, occur in parallel. Thus, we decided to include the time for marshalling the arguments in our throughput figures. We also measured the FTP (raw) communication performance between the client and server for baseline comparison.

In the figure, the three lines saturating at approximately 2MB/s are SPARC and Alpha clients versus J90 Ninf server throughput, indicating that Ninf_call to J90 cannot achieve high throughput due to slow baseline communication performance. The two lines saturating at approximately 3.5 MB/s are SPARC clients versus Alpha Ninf server, and those saturating at around 6 MB/s are when the client and the server are the same architecture, UltraSPARCs and Alphas. In all cases, we see that we are attaining nearly the same throughput as those for FTP (Table 2); thus various communication overhead such as XDR marshalling is not affecting performance significantly.

4 Multi-client LAN/WAN Benchmarks

In global computing, Ninf servers will be processing requests from multiple clients making Ninf_calls simultaneously. In order for the Ninf system (and other global computing systems) to operate effectively, the average processing time should not abnormally increase, and server throughput must be maintained. It is also necessary to measure and identify the feasibility and the possible bottlenecks in WAN.

Under these observations, we performed extensive benchmarks using multiple clients in both LAN and WAN settings. We again employed the Linpack Benchmark and NASPar EP.

4.1 Multi-client LAN/WAN Benchmarking Environment

As a model client program, we employ a routine which calls either the Linpack (sgetrf and sgetrs) or the EP routine repeatedly. We assume that each client performs a Ninf_call on the interval of seconds with probability . We also assume that the number of clients and the matrix size are fixed at and during the benchmark, respectively. We set the other parameters to be , and .

• For LAN benchmarks, we used J90 and SuperSPARC SMP as the computing server, and nodes on an Alpha WS cluster as clients (Figure 2).
• For single-site WAN benchmarks, we also used J90 as the server, and employed 8 to 16 nodes from the SuperSPARCs at the Ochanomizu University located approximately 60km away. The FTP throughput between the client and the server was measured to be approximately 0.17MB/s.
• Multi-site WAN benchmark environment will be covered in Section 4.2.3.

We measured the server's processor utilization, load average, and for each client Ninf_call task, we measured the throughput and various timings: time of task submission , time when the Ninf_call task was accepted at the server , time when the corresponding Ninf executable was invoked , and the time at which Ninf_call was completed . Here, we define the response time and the waiting time of a Ninf_call to be as follows:

As stated in Section 1, there is an option as to how to execute the multiple client requests: 1) distribute the computing resources amongst different client requests in a task parallel manner, or 2) allocate all the processors to each client task in a data parallel manner to be processed in sequence. If the overhead of switching multiple parallel, computationally intensive tasks is high, performance degradation of the latter would be significant. We also note that most typical non-numerical server tasks (such as WWW HTTPD service) take the former approach. To investigate the tradeoffs for J90, the task parallel version has one PE serve each Ninf_call up to 4 parallel tasks, and the data-parallel version employs a optimally vectorized and parallelized version with simultaneous execution on 4 PEs for each Ninf_call, invoked in sequence.

4.2 Multi-client Linpack Benchmark Result

4.2.1 LAN Linpack Benchmark Results

Tables 3 and 4 show the performance of 1-PE and 4-PE executions. The increase in both increases the server load and lowers communication throughput, and as a result we see decreasing Ninf_call performance in both 1-PE and 4-PE versions. Larger variance in the 4-PE version is due to all the PEs being utilized for Linpack execution. When increases, both CPU utilization and load average increase, the former to saturation. Even in such as case, the J90 Ninf server continued to work flawlessly with no dramatic drop in performance (avoiding anomalies such as thrashing, even for , with max. load average 30 for the 4-PE version).

Figure 7 shows the average client-observed performance of Ninf_call. The XY-axis denote problem size and the number of clients , while the Z-axis indicate the performance Mflops. Compared to the 1-PE version, the 4-PE version exhibits higher load and utilization. On the other hand, the 4-PE version exhibits substantial performance edge for a small , while there is very little performance edge for the 1-PE version for a larger . We analyze that this is due to the following reasons: 1) because the numerical core of the 4-PE version is significantly faster due to high vectorization, there is very little loss in average execution time even if all 4 PEs are occupied for a short period, 2) switching parallel tasks on J90 poses small relative overhead compared to large-grain computing tasks, and 3) even though Linpack computation is serialized, communication with clients could be overlapped. In addition, although response and waiting time fluctuated greater for the 4-PE version, their average values did not differ considerably depending on , , or number of processors. We thus conclude that, using the optimized parallel library on the J90 would be appropriate for LAN execution, relieving the library providers from preparing multiple versions.

We also ran the Linpack benchmark on a SuperSPARC SMP server with 16 nodes (Figure 2 and Table 5.) For 1-PE versions, compared to the J90, its performance is more resilient to increase in , and so are response time and waiting time. Also, CPU utilization still has not saturated even for =16. On the other hand, recent preliminary measurements show that highly-multithreaded versions exhibit notable slowdown as c increases (e.g., when number of threads = 12). We speculate the reasons to be as follows: 1) SuperSPARC SMP is less I/O bound compared to J90 because of lower numerical performance, and 2) Solaris 2.5 on SPARC SMP is optimized for handling multiple client requests in server-client situations, and do not co-schedule multiple threads well, resulting in various thread-switching overhead, including cache and TLB misses.

Tables 6 and 7 show the WAN Linpack performance for 1-PE and 4-PE executions. As increases, the ratio between computation vs. communication increases as well, improving the Ninf_call performance as in LAN. On the other hand, when increases, the amount of data sent between the network connecting the two sites increases, causing severe drop in communication throughput. Because of this, the number of Ninf_calls effectively decrease, and as a result, server CPU utilization and load average remains low even for = 16. Thus, under current Internet network throughput, it is difficult for the Ninf computing server to process numerous client requests emanating from the same site (or, at least sharing the same link to the backbone) for communication intensive tasks, requiring a good load balancing algorithm that not only takes into account server load such as is done for NetSolve [6], but also network traffic and global allocation of tasks.

Figure 8 shows the average client-observed performance of Ninf_call under WAN for task- (1-PE) and data-parallel (4-PE) executions. It exhibited almost the same characteristics as LAN; in fact, even when is large, because the server performance has not saturated, the 4-PE versions exhibited better performance. The same is true for response times and waiting times. Thus, we observe that, it is preferable to use the optimized library versions for WAN clients as well.

4.2.3 Multi-site WAN Linpack Benchmark Results

In global computing, in practice clients will be located at different sites in WAN. If simultaneous communication from multiple WAN sites could achieve their aggregate bandwidth so that server load could be maintained high, then global computing would be feasible for communication intensive applications; otherwise, if performance degrades just as the single-site WAN case, then global computing would have smaller market under the current Internet.

For multi-site WAN benchmark, we prepared clients at 4 different University sites in Japan, namely, Ochanomizu University (Ocha-U), University of Tokyo (U-Tokyo), Nagoya Institute of Technology (NITech), and Tokyo Institute of Technology (TITech), while the server is the J90 at ETL. The sites are connected to different backbones as we illustrate in Figure 9. The Linpack routines registered at the J90 Ninf server is the 4-PE version. Figure 10 shows the results.

We can observe that, aggregate communication throughput from multiple sites are substantially higher than that from a single site. In fact, between Ocha-U and ETL, multi-site communication performance deteriorated only by 9%~18% when = 1 (total 4 clients in WAN), and 18%~44% when = 4 (total 16 clients in WAN), maintaining substantially higher bandwidth than when the same number of clients are located at Ocha-U. As a result, CPU utilization and load average is substantially greater for multi-site compared to single-site. Performance-wise, we observe that the rate of performance increase for = 1 is greater than when = 4. Although one plausible explanation would be that the greater number of clients and larger problem size had saturated the computational power of J90, this is not the case as CPU utilization levels off even for = 4 (approx. 27%~34%, not shown in the figure). Because throughput steadily declines for = 4, we believe that network bandwidth saturation is again the cause.

Still, our initial conclusion that, 1) in WAN settings, for communication intensive tasks, point-to-point bandwidth between the client and the server is the dominant factor in determining client-observed performance (and not the current load average of the server), and 2) in general, client Ninf_calls should be distributed to multiple Ninf servers located at different network sites, avoiding concentration of tasks from a single site or those that share the same backbone, in order to maintain performance; otherwise, performance quickly deteriorates significantly as number of clients increase.

4.3 LAN/WAN EP Benchmark Results

EP(An embarrassingly parallel benchmark) is one of the kernel programs in the NAS Parallel Benchmark[7], performing (random-number) Monte-Carlo simulations.

For Ninf benchmarking, we execute the random number generation on the Ninf server. Communication complexity is O(1), namely, problem-size independent. Computational complexity is proportional to the number of random numbers generated, and becomes for trials. Therefore, The execution performance becomes:

where is the time taken for the entire Ninf_call. In each benchmark run, we execute trial samples for each PE, and execute them in a task-parallel manner on the 4-processor J90.

4.3.1 EP Benchmark Results 

Table 8 shows the EP benchmark performance under LAN and single-site, multi-client WAN execution. Because of task-parallel execution, in both LAN and WAN cases Ninf_call performances are almost equivalent and are sustained up to = 4 clients. The performances decline as expected when further increases, but the server utilization remains approximately 100%. This is due to very low communication requirements of EP, and the Ninf_call performance mainly reflects server computing performance. We strongly expect that the result will hold for the multi-site, multi-client WAN case. Thus, for this class of applications such as parallel rendering/imaging, and parameter sensitivity analysis, global computing can now be considered quite feasible.

We also benchmarked automated load balancing using the Ninf metaserver, in order to evaluate its overhead. Because Ninf distributes per each Ninf_call, task-parallel execution of EP would be as follows:

Ninf_transaction_begin();
for (i = 1; i <= numprocs(); i++) {
Ninf_call("ep", ...);
}
Ninf_transaction_end();

Because there are no data-dependencies between multiple Ninf_calls, they will all be scheduled and executed in a task-parallel manner. Given processors, the effective Ninf_call performance would be:

In the benchmark, we employed a 32-processor Alpha cluster in LAN, with each node becoming a Ninf computing server. Figure 11 shows the results. For larger number of trials (class A) and (class B), we achieve almost linear speedup; however, for (sample), we observe significant slowdown; this is because the prototype Metaserver is written in Java, and the overhead of scheduling and distributing Ninf_call has become apparent compared to smaller problem size due to parallel distribution of the EP task. We expect that result will hold for WAN cases from the results of the previous benchmarks. We also conducted benchmarks with DOS (Density-Of-States) calculation, which is an EP-style practical application in computational chemistry, and came up with similar results.

5 Discussions

5.1 Guaranteeing Performance in Multi-Client Global Computing

Performance per each client under multi-client situation cannot be guaranteed irrespective of computing server performance. The same holds true when a single client makes multiple Ninf_calls to the same server. Although it is possible to restrict the number of remote clients, standard TCP-based RPC-protocols require clients and servers to stay connected, as is with current Ninf-RPC. In order to guarantee per-user performance, an alternative is to modify Ninf_call to become a two-phase transaction, where remote argument transfer takes place in the first phase, whereupon the communication is terminated, and after the server computation is over, the client is notified so that it may receive the results in the second phase. We have already implemented such a two-phase protocol for database queries in Ninf [8]. A similar approach is to submit jobs to an intermediate broker server as is with Javelin[9].

As another issue, with the current Internet lacking sufficient bandwidth or bandwidth reservation protocols, communication would become the bottleneck rather than the server's underutilized computing power as we have observed. To resolve this, we are planning to extensively employ the metaserver, which will effectively schedule computation utilizing various performance and communication parameters. In particular, IDL and server execution trace will give us effective information for predicting the communication transfer time versus computing time, making it possible to assign communication- and computation-intensive tasks to appropriate servers.

5.2 Server Job-handling Methodology

Under multi-client setting, we need to improve average response time for Ninf_call, and increase CPU utilization. However, the current Ninf server merely fork & execs a Ninf executable in a First-Come-First-Served (FCFS) manner, causing longer response time and possibly lower CPU utilization. By predicting the computation and communication time of a Ninf_call task using IDL and server trace information, we could perform Shortest-Job-First (SJF) scheduling, improving the response time and utilization considerably. Also, as we have noted, for SMPs, co-scheduling of jobs would be effective in increasing overall system performance, while there would be some sacrifice in system response time.

5.3 Multi-Job Scheduling for MPP Servers

Our benchmark utilized 4 processors for J90 and 16 processors for SPARC SMP. As processor numbers increase, we expect the tradeoffs of various processor assignments to Ninf_call could become different. In particular, task switching of large SPMD tasks could be expensive; in such a case, task-parallel execution could be more efficient. To resolve this, we could extend Ninf IDL to call routines that utilize different number of PEs depending on the problem size.

Additionally, in such a case simple FCFS scheduling may not be the most effective scheduling policy, causing many processors to become idle. To overcome this drawback, we could employ more suitable algorithms such as Fit Processors First Served (FPFS) or Fit Processors Most Processors First Served (FPMPFS)[10]. Still, we need to investigate further which algorithm would be more appropriate for global computing systems such as Ninf.

6 Related Works

The Remote Computation System (RCS)[11] at ETH provides a RPC facility which unifies the interfaces to multiple supercomputers. Because the communication layer is based on PVM it is not well-suited for global computing, and the API lacks the flexibility provided by Ninf.

NetSolve [6] provides almost the same basic API as Ninf_call, and achieves load-balancing with a daemon process called Agents. There are various technical differences between Ninf and NetSolve; examples for Ninf are availability of parallel transactions for load distribution and fault tolerance, while NetSolve facilitates description of computational complexity in library IDL. Also, database server APIs are unique to Ninf. We do expect, however, the results of our benchmark work will also hold for NetSolve, and will have important implications for similar systems. For example, current NetSolve attempts to perform load balancing solely on server load average information; as we have seen, this might partially work for LAN situations, but would not scale to WAN settings.

Legion [12] is a powerful global computing system based on the model of distributed object. A client user distributes his programs across the network by programming with a parallel object-oriented language Mentat [13]. The difference between Mentat and other systems is that it assumes a relatively closed system while Ninf/NetSolve/RCS re-use existing programming languages (although it is possible in Legion to call external programs with wrappers.) Also, Legion assumes more finer-grained computing objects. Because of this, we have to be careful in observing whether our benchmark results will hold for Legion, but we expect that many of them will, as it is not possible to achieve good performance in global computing in general without being relatively coarse-grained.

There are other global computing projects that focus on different level of system architecture, such as Globus/Nexus[4], or whose focus is more on portability and other issues, such as Javelin[9].

7 Conclusion and Future Work

We performed an extensive performance analysis of Ninf, a global computing system for high-performance computing. We ran several benchmarks with different communication/computation characteristics on a variety of combinations of clients and servers, in their performance, architecture, etc., under LAN, single-site WAN, and multi-site WAN situations. We conclude with the followings:

• The current Ninf system (and, other systems such as NetSolve) would function in both LAN and WAN situations. The use of optimized parallel library would be sufficient for setting up numerous global computing servers all around the world, although for SMPs we do need to determine the optimal number of threads given the number of clients.
• On the other hand, there must be a variety of improvements that must be made, as we have indicated, both in communication architecture of the client-server, and task scheduling/load distribution, especially in WAN setting.
• In LAN setting, computing server performance dictates overall performance, while in WAN limitation of communication throughput is currently more significant. In particular, multiple client requests from a single site could quickly saturate the network for communication-intensive applications. However, we expect that in practice multiple client requests will be issued from different sites, in which case performance will depend of Internet network topologies. In fact, for EP-style applications, current Ninf or other systems such as NetSolve would be quite feasible.

Finally, although we have performed extensive benchmarks on two application core examples, we still do need to perform even more benchmarks using other types of practical applications, and different WAN configurations. However, on the Internet it is quite difficult to perform large-scale benchmarks with reproducible results. One current plan we have is to build a global computing simulator for Ninf, on which we could readily test different client network topologies under various communication and other parameters. Still, we do need to conduct further application benchmarks to determine parameters in the underlying global computation model for the Ninf simulator.

We also need to make our Ninf system more robust so that it could be readily distributed for global public usage, upon which many applications should be written, based on which we could gain more knowledge on the behavior of global computing systems in general. We also hope to collaborate with other efforts in global computing so that we could share as much of the technical results and infrastructures as possible.

Footnotes

¹ Although the current Ninf client API only supports matrices, we plan to extend the IDL and the interpreter to handle arbitrary user-defined objects.

References

[1] http://phase.etl.go.jp/ninf/

[2] M. Sato, H. Nakada, S. Sekiguchi, S. Matsuoka, U. Nagashima, and H. Takagi. Ninf: A Network based Information Library for a Global World-Wide Computing Infrastructure. In Proceedings of HPCN'97 (LNCS-1225), pages 491-502, 1997.

[3] H. Nakada, M. Sato, and S. Sekiguchi. An overview of the RPC System Implementation for Ninf, Technical Report TR95-28, Electrotechnical Laboratory, 1995.

[4] I. Foster, and C. Kesselman. Globus: A Metacomputing Infrastructure Toolkit, International Journal of Supercomputer Applications (to appear), 1997.

[5] Oguni, editor. Matrix Computing Software. Maruzen, 1991.

[6] H. Casanova, and J. Dongarra. NetSolve: A Network Server for Solving Computational Science Problems. In Proceedings of Supercomputing '96, 1996.

[7] http://science.nas.nasa.gov/Software/NPB/

[8] M. Iioka et al. Global Numerical Information Database Server System for Ninf. In Proceedings of 13th Workshop on Object-Oriented Computing, Hakone, Japan, 1997.

[9] P. Cappello, B. Christiansen, M. F. Ionescu, M. O. Neary, K. E. Schauser, and D. Wu. Javelin: Internet-Based Computing Using Java. In 1997 ACM Workshop on Java for Science and Engineering Computation (submitted), 1997.

[10] K. Aida, H. Kasahara, and S. Narita. Scheduling Scheme of Parallel Jobs on a Multiprocessor System, Supercomputing '97 (submitted to Technical Paper), 1997.

[11] P. Arbenz, W. Gander, and M. Oettli. The Remote Computational System, High-Performance Computation and Network. Volume 1067 of Lecture Note in Computer Science, pages 662-667, Springer, 1996.

[12] A. S. Grimshaw, W. A. Wulf, J. C. French, A. C. Weaver, and P. F. Reynolds Jr. Legion: The Next Logical Step Toward a Natiowide Virtual Computer. Technical Report CS-94-21, University of Virginia, 1994.

[13] A. S. Grimshaw. Easy to Use Object-Oriented Parallel Programing with Mentat, IEEE Computer, pages 39-51, 1993.