Invoke Server Developer's Manual

This document describes how to develop a Ninf-G Invoke Server.

1. Introduction

A Ninf-G Client invokes a Ninf-G Executable on the server machine when a function requiring initialization of function/object handles, such as grpc_function_handle_init(), is called. Ninf-G, Version 2, implements the remote process invocation using the Globus Toolkit's Pre-WS GRAM feature. Implemented using the Globus API, the invocation mechanism has been embedded in Ninf-G. In order to utilize other systems, such as WS GRAM, UNICORE, or Condor for remote process invocation, Ninf-G, Version 4, implements the invocation mechanism as a separate module called "Invoke Server." This design enables users and developers to implement and add a new Invoke Server that can utilize any job invocation mechanism.

Ninf-G Version 4.2.0 includes the following Invoke Servers:

1.1 Overview of a typical client application

Here is a typical flow of a Ninf-G Client application:

Invoke Server is required to implement initialization and finalization of the function/object handles which are described in steps (2) and (4).

1.2 Requirements for underlying middleware

The only requirement for underlying middleware is that the middleware must be capable of remote process invocation. Examples of such middleware include the Globus Toolkit Pre-WS GRAM, Globus Toolkit WS GRAM, Unicore, Condor, and SSH.

1.3 Implementation overview

Invoke Server is an adapter for the underlying middleware and it handles requests from a Ninf-G Client. Invoke Server analyzes and processes the request sent from the Ninf-G Client and replies to the Ninf-G Client. For example, if Invoke Server receives a JOB_CREATE request from the Ninf-G Client, Invoke Server creates a Job ID, returns the Job ID to the Ninf-G Client, and invokes the job processes called for in the request.

Invoke Server can be implemented using any language. The details of the protocol existing between the Ninf-G Client and Invoke Server are described in Section 2.

1.4 Execution flow

This section describes a sample RPC flow to a server called serverA via the Invoke Server, IS_SAMPLE.

The following figure illustrates the interaction between the Ninf-G Client, Invoke Server, and the Ninf-G Executable.

Interaction of Invoke Server

Figure 1: Interaction between the Ninf-G Client, Invoke Server and the Ninf-G Executable

2. Specifications of Invoke Server

This section describes a detailed overview of Invoke Server and the protocol existing between a Ninf-G Client and Invoke Server.

2.1 Detailed overview of Invoke Server

  1. Invoke Server is invoked when a Ninf-G Client initializes a function/object handle on the remote server which Ninf-G Client is configured to use with Invoke Server.

  2. The maximum number of jobs per Invoke Server is limited. If the number of jobs exceeds the limit, a new Invoke Server is invoked.

  3. Invoke Server exits the process if it receives an EXIT request from the Ninf-G Client. This request is sent when the Ninf-G Client calls grpc_finalize(). Invoke Server also exits the process if it is managing the maximum number of jobs and all jobs are terminated.

  4. The Ninf-G Client and Invoke Server communicate using three pipes, created by the Ninf-G Client when the Invoke Server is invoked.

  5. Ninf-G Client does not wait for the termination of Invoke Server after the Ninf-G Client sends an EXIT request to Invoke Server.

  6. If the Ninf-G Client exits abnormally, the pipes will be disconnected. When Invoke Server detects that the pipes have been disconnected, Invoke Server must cancel all jobs and exit the process.

  7. Invoke Server is implemented as a Unix executable or script file which should be located in the ${NG_DIR}/bin directory. It can be located in another directory if Invoke Server is supplied with an absolute path to the executable file.

  8. The file names used with Invoke Server must follow the naming convention of "ng_invoke_server" + suffix, where the suffix corresponds to rules for the underlying middleware used for remote process invocation.

  9. Log file for Invoke Server can be specified as an optional argument of the Invoke Server command.


    -l [Log file name]

    If this option is specified, Invoke Server outputs logs to the file specified by this argument. Otherwise, logs are not recorded.

2.2 Protocol between a Ninf-G Client and Invoke Server

2.2.1 Overview

A Ninf-G Client and Invoke Server exchange three types of messages, Request, Reply, and Notify. A Request message is sent from a Ninf-G Client to Invoke Server. Reply and Notify messages are sent from Invoke Server to the Ninf-G Client. The Ninf-G Client assumes that a Reply message must be returned from Invoke Server when the Ninf-G Client sends a Request message. A Notify message is used to send messages from Invoke Server to the Ninf-G Client asynchronously. Three different pipes are used for sending these three types of message.

Requeststdin Ninf-G---->Invoke
Replystdout Client<----Server
Notifystderr <----

2.2.2 Protocol

All messages are sent as plain text. The Return code (<RET>) is 0x0d0a. The Return code is a delimiter that determines the unit of messages. A Job ID is generated by Invoke Server. Request

Four Request messages, JOB_CREATE, JOB_STATUS, JOB_DESTROY, and EXIT are supported.

  4. EXIT Reply

Invoke Server must send a Reply message to a Ninf-G Client if Invoke Server receives a Request message from that Ninf-G Client.

The reply to JOB_CREATE, JOB_DESTROY, and EXIT messages is:

[S   | F <Error String>]<RET>

where S is sent in case of Success. Otherwise, F is returned, followed by <Error String>.

The reply to a JOB_STATUS message is:

[S <Status>  | F <Error String>]<RET>

Where <Status> is denoted as:


Each status indication indicates the status such that: Notify

A Notify message is used to send an asynchronous message from Invoke Server to a Ninf-G Client. Two types of Notify message are provided.


This section describes the details of a JOB_CREATE Request.

Appendix A. How to specify the Invoke Server

Invoke Server is specified by the Ninf-G Client using a Client configuration file.

A.1. How to specify Invoke Server

Invoke Server is specified by using the invoke_server attribute in the <SERVER> section.

invoke_server [type]

Type specifies the type of the Invoke Server, such as GT4py or UNICORE.

A.2. How to pass information to Invoke Server

Invoke Server may require options for its execution. Such options can be specified by an option attribute in the <INVOKE_SERVER> section or by an invoke_server_option attribute in the <SERVER> section.

option [String]
invoke_server_option [String]

Multiple attributes can be specified in the <SERVER> or <INVOKE_SERVER> sections.

A.3. Polling interval

Invoke Server must check the status of jobs, and this may be implemented using polling. The polling interval can be specified by the status_polling attribute in the <INVOKE_SERVER> section.

status_polling [interval (seconds)]

A.4. Logfile

The filename of the Invoke Server's execution log can be specified by the invoke_server_log attribute in the <CLIENT> section.

invoke_server_log [filename]

If this attribute is specified, Invoke Server outputs logs to a file with the specified filename and file type of that Invoke Server.

The log_filePath attribute in the <INVOKE_SERVER> section can be used to specify a log file for a specific Invoke Server.

log_filePath [Log file name]

A.5. Maximum number of jobs per Invoke Server

The maximum number of jobs per Invoke Server can be limited by the max_jobs attribute in the <INVOKE_SERVER> section. If the number of requested jobs exceeds this value, the Ninf-G Client invokes a new Invoke Server and requests that Invoke Server to manage the new jobs.

max_jobs [maximum number of jobs]

A.6. How to specify the path of the Invoke Server

If Invoke Server is not located in a pre-defined directory, the path attribute in <INVOKE_SERVER> can be used to specify the path of the Invoke Server.

path [path of the Invoke Server]

Appendix B. Miscellaneous Information

B.1. Job Timeout

The Job Timeout function is managed by the Ninf-G Client. Invoke Server is not responsible for the timeout.

B.2. Redirect stdout/stderr is implemented using files

Redirect stdout/stderr is implemented using files.

last update : $Date: 2006/09/20 04:57:42 $