| Chapter 9 |
Interprocess Communication under LISP |
|
by William Lott and Bill Chiles
CMUCL offers a facility for interprocess communication (IPC)
on top of using Unix system calls and the complications of that level
of IPC. There is a simple remote-procedure-call (RPC) package build
on top of TCP/IP sockets.
The remote package provides simple RPC facility including
interfaces for creating servers, connecting to already existing
servers, and calling functions in other Lisp processes. The routines
for establishing a connection between two processes,
create-request-server and connect-to-remote-server,
return wire structures. A wire maintains the current state of
a connection, and all the RPC forms require a wire to indicate where
to send requests.
| 9.1.1 |
Connecting Servers and Clients |
|
Before a client can connect to a server, it must know the network address on
which the server accepts connections. Network addresses consist of a host
address or name, and a port number. Host addresses are either a string of the
form VANCOUVER.SLISP.CS.CMU.EDU or a 32 bit unsigned integer. Port
numbers are 16 bit unsigned integers. Note: port in this context has
nothing to do with Mach ports and message passing.
When a process wants to receive connection requests (that is, become a
server), it first picks an integer to use as the port. Only one server
(Lisp or otherwise) can use a given port number on a given machine at
any particular time. This can be an iterative process to find a free
port: picking an integer and calling create-request-server. This
function signals an error if the chosen port is unusable. You will
probably want to write a loop using handler-case, catching
conditions of type error, since this function does not signal more
specific conditions.
[Function]
wire:create-request-server port &optional on-connect
create-request-server sets up the current Lisp to accept
connections on the given port. If port is unavailable for any
reason, this signals an error. When a client connects to this port,
the acceptance mechanism makes a wire structure and invokes the
on-connect function. Invoking this function has a couple of
purposes, and on-connect may be nil in which case the
system foregoes invoking any function at connect time.
The on-connect function is both a hook that allows you access
to the wire created by the acceptance mechanism, and it confirms the
connection. This function takes two arguments, the wire and the
host address of the connecting process. See the section on host
addresses below. When on-connect is nil, the request server
allows all connections. When it is non-nil, the function returns
two values, whether to accept the connection and a function the
system should call when the connection terminates. Either value may
be nil, but when the first value is nil, the acceptance mechanism
destroys the wire.
create-request-server returns an object that
destroy-request-server uses to terminate a connection.
[Function]
wire:destroy-request-server server
destroy-request-server takes the result of
create-request-server and terminates that server. Any
existing connections remain intact, but all additional connection
attempts will fail.
[Function]
wire:connect-to-remote-server host port &optional on-death
connect-to-remote-server attempts to connect to a remote
server at the given port on host and returns a wire
structure if it is successful. If on-death is non-nil, it is
a function the system invokes when this connection terminates.
After the server and client have connected, they each have a wire
allowing function evaluation in the other process. This RPC mechanism
has three flavors: for side-effect only, for a single value, and for
multiple values.
Only a limited number of data types can be sent across wires as
arguments for remote function calls and as return values: integers
inclusively less than 32 bits in length, symbols, lists, and
remote-objects (see section 9.1.3). The system sends symbols
as two strings, the package name and the symbol name, and if the
package doesn't exist remotely, the remote process signals an error.
The system ignores other slots of symbols. Lists may be any tree of
the above valid data types. To send other data types you must
represent them in terms of these supported types. For example, you
could use prin1-to-string locally, send the string, and use
read-from-string remotely.
[Macro]
wire:remote wire {call-specs}*
The remote macro arranges for the process at the other end of
wire to invoke each of the functions in the call-specs.
To make sure the system sends the remote evaluation requests over
the wire, you must call wire-force-output.
Each of call-specs looks like a function call textually, but
it has some odd constraints and semantics. The function position of
the form must be the symbolic name of a function. remote
evaluates each of the argument subforms for each of the
call-specs locally in the current context, sending these
values as the arguments for the functions.
Consider the following example:
(defun write-remote-string (str)
(declare (simple-string str))
(wire:remote wire
(write-string str)))
The value of str in the local process is passed over the wire
with a request to invoke write-string on the value. The
system does not expect to remotely evaluate str for a value
in the remote process.
[Function]
wire:wire-force-output wire
wire-force-output flushes all internal buffers associated
with wire, sending the remote requests. This is necessary
after a call to remote.
[Macro]
wire:remote-value wire call-spec
The remote-value macro is similar to the remote macro.
remote-value only takes one call-spec, and it returns
the value returned by the function call in the remote process. The
value must be a valid type the system can send over a wire, and
there is no need to call wire-force-output in conjunction
with this interface.
If client unwinds past the call to remote-value, the server
continues running, but the system ignores the value the server sends
back.
If the server unwinds past the remotely requested call, instead of
returning normally, remote-value returns two values, nil
and t. Otherwise this returns the result of the remote
evaluation and nil.
[Macro]
wire:remote-value-bind wire ({variable}*) remote-form
{local-forms}*
remote-value-bind is similar to multiple-value-bind
except the values bound come from remote-form's evaluation in
the remote process. The local-forms execute in an implicit
progn.
If the client unwinds past the call to remote-value-bind, the
server continues running, but the system ignores the values the
server sends back.
If the server unwinds past the remotely requested call, instead of
returning normally, the local-forms never execute, and
remote-value-bind returns nil.
The wire mechanism only directly supports a limited number of data
types for transmission as arguments for remote function calls and as
return values: integers inclusively less than 32 bits in length,
symbols, lists. Sometimes it is useful to allow remote processes to
refer to local data structures without allowing the remote process
to operate on the data. We have remote-objects to support
this without the need to represent the data structure in terms of
the above data types, to send the representation to the remote
process, to decode the representation, to later encode it again, and
to send it back along the wire.
You can convert any Lisp object into a remote-object. When you send
a remote-object along a wire, the system simply sends a unique token
for it. In the remote process, the system looks up the token and
returns a remote-object for the token. When the remote process
needs to refer to the original Lisp object as an argument to a
remote call back or as a return value, it uses the remote-object it
has which the system converts to the unique token, sending that
along the wire to the originating process. Upon receipt in the
first process, the system converts the token back to the same
(eq) remote-object.
[Function]
wire:make-remote-object object
make-remote-object returns a remote-object that has
object as its value. The remote-object can be passed across
wires just like the directly supported wire data types.
[Function]
wire:remote-object-p object
The function remote-object-p returns t if object
is a remote object and nil otherwise.
[Function]
wire:remote-object-local-p remote
The function remote-object-local-p returns t if
remote refers to an object in the local process. This is can
only occur if the local process created remote with
make-remote-object.
[Function]
wire:remote-object-eq obj1 obj2
The function remote-object-eq returns t if obj1 and
obj2 refer to the same (eq) lisp object, regardless of
which process created the remote-objects.
[Function]
wire:remote-object-value remote
This function returns the original object used to create the given
remote object. It is an error if some other process originally
created the remote-object.
[Function]
wire:forget-remote-translation object
This function removes the information and storage necessary to
translate remote-objects back into object, so the next
gc can reclaim the memory. You should use this when you no
longer expect to receive references to object. If some remote
process does send a reference to object,
remote-object-value signals an error.
The wire package provides for sending data along wires. The
remote package sits on top of this package. All data sent
with a given output routine must be read in the remote process with
the complementary fetching routine. For example, if you send so a
string with wire-output-string, the remote process must know
to use wire-get-string. To avoid rigid data transfers and
complicated code, the interface supports sending
tagged data. With tagged data, the system sends a tag
announcing the type of the next data, and the remote system takes
care of fetching the appropriate type.
When using interfaces at the wire level instead of the RPC level,
the remote process must read everything sent by these routines. If
the remote process leaves any input on the wire, it will later
mistake the data for an RPC request causing unknown lossage.
When using these routines both ends of the wire know exactly what types are
coming and going and in what order. This data is restricted to the following
types:
-
8 bit unsigned bytes.
- 32 bit unsigned bytes.
- 32 bit integers.
- simple-strings less than 65535 in length.
[Function]
wire:wire-output-byte wire byte
[Function]
wire:wire-get-byte wire
[Function]
wire:wire-output-number wire number
[Function]
wire:wire-get-number wire &optional
signed
[Function]
wire:wire-output-string wire string
[Function]
wire:wire-get-string wire
These functions either output or input an object of the specified
data type. When you use any of these output routines to send data
across the wire, you must use the corresponding input routine
interpret the data.
When using these routines, the system automatically transmits and interprets
the tags for you, so both ends can figure out what kind of data transfers
occur. Sending tagged data allows a greater variety of data types: integers
inclusively less than 32 bits in length, symbols, lists, and remote-objects
(see section 9.1.3). The system sends symbols as two strings, the
package name and the symbol name, and if the package doesn't exist remotely,
the remote process signals an error. The system ignores other slots of
symbols. Lists may be any tree of the above valid data types. To send other
data types you must represent them in terms of these supported types. For
example, you could use prin1-to-string locally, send the string, and use
read-from-string remotely.
[Function]
wire:wire-output-object wire object &optional cache-it
[Function]
wire:wire-get-object wire
The function wire-output-object sends object over
wire preceded by a tag indicating its type.
If cache-it is non-nil, this function only sends object
the first time it gets object. Each end of the wire
associates a token with object, similar to remote-objects,
allowing you to send the object more efficiently on successive
transmissions. cache-it defaults to t for symbols and
nil for other types. Since the RPC level requires function
names, a high-level protocol based on a set of function calls saves
time in sending the functions' names repeatedly.
The function wire-get-object reads the results of
wire-output-object and returns that object.
| 9.2.3 |
Making Your Own Wires |
|
You can create wires manually in addition to the remote
package's interface creating them for you. To create a wire, you need
a Unix file descriptor. If you are unfamiliar with Unix file
descriptors, see section 2 of the Unix manual pages.
[Function]
wire:make-wire descriptor
The function make-wire creates a new wire when supplied with
the file descriptor to use for the underlying I/O operations.
[Function]
wire:wire-p object
This function returns t if object is indeed a wire,
nil otherwise.
[Function]
wire:wire-fd wire
This function returns the file descriptor used by the wire.
The TCP/IP protocol allows users to send data asynchronously, otherwise
known as out-of-band data. When using this feature, the operating
system interrupts the receiving process if this process has chosen to be
notified about out-of-band data. The receiver can grab this input
without affecting any information currently queued on the socket.
Therefore, you can use this without interfering with any current
activity due to other wire and remote interfaces.
Unfortunately, most implementations of TCP/IP are broken, so use of
out-of-band data is limited for safety reasons. You can only reliably
send one character at a time.
The Wire package is built on top of CMUCLs networking support. In
view of this, it is possible to use the routines described in section
10.6 for handling and sending out-of-band data. These
all take a Unix file descriptor instead of a wire, but you can fetch a
wire's file descriptor with wire-fd.