The index for the Allegro CL Documentation is in index.htm. The documentation is described in introduction.htm.
This document contains the following sections:
1.0 Simple-stream introduction
Starting in release 6.0, Allegro CL has a new streams model using
simple-streams, which formally have no element-type but act in general
as if they have element type (unsigned-byte
8). The new implementation is described in this document.
Release 5.0.1 and earlier used a stream implementation called Gray
streams. That implementation is still supported. It is described
in gray-streams.htm
A simple-stream is created whenever a file is opened (with open) without an element-type specified. If open is called with an element-type specified, a Gray stream is created.
The transition from Gray streams to simple-streams should be easy and transparent unless you have done extensive stream customization.
It is unlikely that users who are not concerned with stream details
will have to concern themselves with this document. Common stream
usages such as opening files for reading and writing) will simply
work as expected. If you want things to work virtually identically to
how they worked in release 5.0.1, see
3.1 How to get a simple-stream and how to get a Gray stream, where it says that you
get 5.0.1 behavior so long as you specify an element-type to
open and that you should use character as
the element type when your 5.0.1 code did not specify one. Do note
that you cannot assume that system-created streams will be Gray
streams in 6.0.
A new kind of stream is introduced, called a simple-stream. This stream is intended to serve as the new preferred alternative to Gray streams, which is what previous versions of Allegro CL used to implement Common Lisp streams functionality. Simple streams are simpler than Gray streams, easier to extend, and faster. Both kinds of stream may co-exist in a lisp, and compatibility is maintained for the standard Common Lisp streams interface, the Gray streams implementation maintains its compatibility with previous Gray versions with minimal source intervention when they have been used.
The Allegro CL simple-stream is the next generation evolving from the stream of the same name in Common Graphics (the windowing systems used by Allegro CL on Windows). The Common Graphics simple-stream was created as a basis for window-based input and output. The new generation of simple-stream is designed to include this window-based I/O and at the same time to retain the speed always expected for non-window based streams. It is also designed to promote further advances in technology requirements such as International Character sets and server-based I/O.
We felt a new stream implementation was needed because of problems we found with the Gray stream design. In this section, we describe these problems and describe the new implementation.
A new class hierarchy of streams with a base-class of
simple-stream
removes the problems inherent
in Gray streams. Simple-streams are more efficient, and are simpler in
concept, making it easier to extend the streams interface by
object-oriented means.
A major simplification in the simple-stream hierarchy over Gray streams is the collapsing of many class distinctions into one:
The Gray streams class hierarchy distinguishes between different kinds of stream usage. The simple-stream hierarchy distinguishes between different kinds of external I/O device or pseudo-device.
This device-level interface is CLOS oriented, but is at a much lower level than the Gray streams implementation level, making it much more efficient in both execution speed and in space.
The Gray streams implementation (which has been in Allegro CL for some
time) will continue to be available in release 6.0. Programs using
customized Gray streams will, therefore, continue to work as in
earlier releases with only minor changes. (Users of Gray streams must
ensure, as described below, that calls to open
have the element-type keyword argument specified -- if it is
unspecified, give it the value character. Users
must also be careful not to assume a stream they did not create is a
Gray stream, and that can be done using synonym streams or by loading
a compatibility package, again as described below in this
section. Gray streams are described in
gray-streams.htm.)
The normal mechanism used to specify a simple-stream is to call open (and thus any callers of open, such as with-open-file) to open the stream without specifying the element-type. This will cause a simple-stream to be created, instead of a Gray stream. A Gray stream will be created if an element-type argument of any kind is given to the open call.
A potential problem arises when legacy code requiring Gray streams
calls open with no
element-type argument. Under CL specification this kind of
open causes the element-type to default to
character. All CL functionality will be compatible
between Gray streams and simple-streams, but if the user was counting
on specific Gray stream functionality in character streams, then the
open call must be changed to
include :element-type 'character as arguments,
which will force a Gray stream.
Another problem arises when Gray streams application code assumes that the stream it is handed is a Gray stream, and thus tries to call stream- methods on it. (Allegro CL names the Gray streams CLOS substrate stream-[cl-function-name], e.g. stream-read-char for read-char.) Allegro CL with the new stream implementation solves this problem by defining its synonym-stream implementation as a Gray stream, but in such a way that all calls via the synonym-stream-symbol are non-Gray-specific. For example, a call to stream-read-char to a synonym-stream will result in a call to read-char on the synonym-stream-symbol. This is slightly slower than dispatching on stream-read-char, but it does provide for compatibility with legacy code. A programmer who doesn't want to rewrite a subsystem to use simple-streams can simply ensure that any stream passed to that subsystem is a synonym stream.
If the stream being manipulated is one that is not easily wrapped as a
synonym-stream, (e.g. *terminal-io*) a second approach is provided in
the form of the module :gray-compat. This module
contains methods on simple-streams for generic functions normally
associated with Gray streams. If, for example, the call
(stream-read-char *standard-input*)
exists in legacy code, then requiring the
:gray-compat module (by evaluating
(require :gray-compat)) defines a method on
stream-read-char for simple-streams, which simply
wraps some argument and return-value processing around a call to
read-char. This again is slower than calling
read-char directly, but provides compatibility for
legacy code.
CL stream-functions read-byte, write-byte, read-char, etc., all distinguish in a trivial manner whether the stream is a Gray or simple stream. If a Gray stream is detected, the associated Gray generic function is called for the stream, so that for example, read-char calls stream-read-char, write-char calls stream-write-char, etc. However, if the stream is determined to be a simple-stream, then the specified lower level functionality for the function is called, which may involve calls to specific device-level functionality. This is described in section 5.1 Implementation of Common Lisp Functions for Simple-Streams below.
Note that this trivial dispatch does not use any CLOS dispatch mechanism, and the functionality that is called for a simple-stream may be inlined in the function. Thus, for example, all write-byte operations for a simple-stream are performed without any function calls, unless the buffer fills up and device-write or device-extend must be called.
A simple-stream has no specific element-type associated with it. Instead, the fundamental unit of transfer for a simple stream that is not a string stream is the octet or 8-bit byte, and all transfers are made at the lowest level with respect to octets. It is up to the implementation to decide how to optimize data transfers for particular situations where data paths are either wider or narrower than 8 bits.
A simple-stream is always buffered. Whereas support is provided for buffering for CL and Gray streams, buffering is not explicitly required in these stream specifications. However, the explicit requirement that simple-streams are buffered allow a simpler and potentially more efficient model. Note that there is no direct interface to simple-stream buffers. The buffering layer resides just below the CL interface level, and the device layer is just below the buffering layer:
User Level Strategy Level Device-level
| |
--------------------
| CL functionality | | |
--------------------
| | | | |
| | -------------------------------------
| | | | |
| | v
| | | | -------------------
| | | Control-character |
| | | | | processing |
| | -------------------
| | | | | | |
| | .--------------- | |
| | | v | | |
| | ------------------ | |
| | | | external-format | | | |
| | | processing | | |
| | | ------------------ | | |
| | | | |
| -------------------. | .------------------ |
| | v v v | |
| --------------- |
| | | Buffering | | |
| --------------- |
| | | | |
| -----------------------. |
--------------------------------------------------. | |
| | v v v
-----------------
| | | Device layer |
-----------------
String streams bypass the external-format (that is, External File Format, as defined in Common Lisp) processing, since their destinations are not really external.
Programmers will work with simple-streams at various levels, wearing one of three different hats at any one time:
There is also a set of functions provided which aid in the implementation of the device layer, and which at the same time are themselves User Level functions. These functions allow the design of encalpsulating streams, where the encapsulating stream's device level becomes the encapsulated stream's user level. These functions are discussed in 10.4 Implementation Helpers for device-read and device-write.
The intended interaction by the user or applications programmer is to work above the buffer level. The user does so by calling standard CL functions. The device-level programmer may define new classes and device-level methods for "drivers" (we are using the word analogously to device drivers that are implemented in operating systems), but even then it is not intended that the user call the device-level methods directly. But note that it is possible to call device-level methods if all of the rules are followed. The strategy-level programmer may design an alternate API that calls the device-level, but it must conform to the requirements that allow the device-level to work properly. The intended role of the API is that it is a thin layer which manipulates its buffer and thus deals with the device layer as little as possible. Such API's are intended to be very fast.
The device level is called that because it provides an underlying implementation that can be specialized to suit particular kinds of stream connections, in a similar manner to a device driver in an operating system.
Only simple-streams provide a device layer; Gray streams do not. The device layer puts the object implementation of a simple-stream at a lower level than the object layer of Gray streams.
The goals of the device layer are:
The device layer is not intended to be called directly, except by strategies for higher-level API interfaces that conform to strategy rules. Such APIs should be very lightweight and fast so that there is no need or temptation to call the device-layer directly. Creators of such higher-level APIs must be especially careful to understand the buffering issues involved, including those described in device-read and device-write.
Note that the device layer can implement whatever kind of connection it is set up to do. Usually this means that it will talk directly to a file handle or file descriptor number. However, the connection can be made to a stream of a different type instead of directly to an operating-system level file. By this means, Java style stream encapsulations can be created by the device-level programmer.
Such encapsulation functionality is done automatically by some functions provided as implementation helpers (ee 10.4 Implementation Helpers for device-read and device-write).
Simple-streams are normally opened with device-open and closed with device-close. device-buffer-length returns the desired length of buffers to be allocated for the stream, if any. device-file-position returns a positive integer that is the current octet (8-bit byte) position of the device represented by its argument stream. device-file-length returns the number of octets (8-bit bytes) in the argument stream if possible.
device-read fills a buffer (if possible) with data from its argument stream. device-clear-input clears any pending input on the device connected to its argument stream. device-write writes from the buffer to the argument stream. device-clear-output clears pending output.
Methods that don't fall under the strict buffer-unaware read-write device methods include device-extend and device-finish-record. Unlike device-read and device-write, these methods may manipulate stream slots, allocate new work spaces, or call out recursively to higher level stream functions. The intention here is to separate the pure fill and flush aspect of device-read and device-write from the more complex aspects of mapping and record-orientation.
The one exception to the buffer-unaware separation in device-read and device-write is when they receive a null buffer argument from the strategy layer, and their start and end arguments are not the same. This will occur if the buffer that would have been passed is the actual buffer of the stream.
Under this circumstance the device-read/device-write method has a
little leeway; it must assume that the null buffer argument refers to
the appropriate buffer in the actual stream, and must retrieve that
argument for use. However, it is free to detach and/or replace the
buffer with another of the same size. Also, in the case of device-read, the length of the
buffer must be used as the end argument, which
will also be nil if the buffer argument is
nil (unless end is also
eql to start). This flagging of the stream's
buffer enables device-read
and device-write methods to
be written that perform advanced buffer-management and asynchronous
read-write operations.
The first subsection describes the implementation of standard Common Lisp functions that deal with streams. Note that the behavior is usually different for Gray streams (where the CL function usually calls an Allegro-CL-specific associated generic functions) and for simple-streams (on which the CL function usually operates directly).
The second subsection describes additional functions that operate on streams, but are specific to Allegro CL.
Given the device interface, we can now describe how standard Common Lisp functions and some related Allegro CL functions are implemented in terms of these driver functions. Because the intention of this section is to provide implementation information, but not to describe how to use the functions, usage details such as argument lists are not provided.
See open for the ANSI description.
For both Gray and simple streams, open effectively turns into a call to make-instance of a stream
class. Additionally, for simple-streams, a shared-initialize after method calls device-open to actually establish
the connection with the external device or file. If the device-open call then fails and thus
returns nil, then device-close is called immediately with a true
abort argument.
As it did in previous versions of Allegro CL, cl:open has an &allow-other-keys specification, and an &rest argument. This &rest argument forms the basis of the make-instance initargs when it is called via apply.
A special case exists for an open with
:direction :probe: this case is not a normal open
and does not actually result in a connection of any kind being
made. Instead, make-instance is called to make an
instance of probe-simple-stream.
See close for the ANSI description.
The Gray stream system in Allegro CL implements close as a generic function, which is perfectly legal according to CL, which defines close as a function (i.e. a generic function is indeed a function). However, a generic function implies a specialization capability that does not exist for simple-streams; simple-stream specializations should be on device-close. Besides Gray streams, close can be specialized on streams that are neither Gray or simple-streams. One example of this is Allegro CL's passive socket connection. Because of this, close remains a generic function, but for simple streams is treated as if non-generic, that is simple-streams should not specialize on close, but should specialize on device-close instead. The method for simple-streams simply calls device-close precisely once, and a method for fundamental stream (the top-level Gray stream class) does what it did for previous versions of Allegro CL: it breaks the connection and sets a closed-flag in the stream.
If the abort keyword argument is true, any buffers are cleared without being flushed. If abort is false, then any unflushed buffers are forced out to the device before closing.
See read-byte for the ANSI description.
For a Gray stream, read-byte calls stream-read-byte.
Otherwise: If the stream's buffer is empty, an attempt is made to fill the buffer by calling device-read with the blocking argument set to true. If -1 was returned, then we are at eof; either eof-value is returned or else an end-of-file error occurs.
If the stream's buffer is now not empty, the next octet (8-bit byte) is extracted from the buffer and returned.
See read-char for the ANSI description.
For a Gray stream, read-char calls stream-read-char.
Otherwise: The external format is called to accumulate (as if using read-byte) as many octets (8-bit bytes) as is necessary to form a character. If an end-of-file is generated by any of the read-bytes, eof processing is done depending on the eof arguments.
If the character that results is a control character and the
control-in table has a function for that character, then it is a
function with two arguments (the stream and the character) which is
called to interpret the control-character at this time. If the
control-in function returns, it returns either a character which is
processed normally, or nil, which is
interpreted as an eof and eof processing is done. Note that the
control-in handler must not try to do any reading from the stream at
all; the intention for the control-in handler is to translate an
already-received character to another, or to perform an operation and
return a character. For ligatures and other multiple-character inputs,
a composing external-format should be used or created, or else an
encapsulation created for such translations.
If we got this far, the character length is recorded for unread-char and the character is returned.
Note that if eof occurs while reading a character, the actions taken by read-char depend on the external-format. The default action, and by far the most common, is to do eof processing. However, the external format may decide to return a character (saved from a previous read-char) or to generate an error.
See unread-char for the ANSI description.
For a Gray stream, unread-char calls stream-unread-char.
Otherwise: if the unread-char character-length is set, then place the buffer and file position back to that and unset the unread-char length. Error if the unread-char character-length is not set.
See read-char-no-hang for the ANSI description.
For a Gray stream, read-char-no-hang calls stream-read-char-no-hang.
Otherwise: The external format is called to accumulate (as if using excl::read-byte-no-hang) as many octets (8-bit bytes) as is necessary to form a character. If an end-of-file is generated by any of the byte reads, eof processing is done depending on the eof arguments.
If the character is a control character and the stream-class specifies interpretation of such characters, it is performed at this time, which may include eof-processing for a control-D.
If we got this far, the character length is recorded for unread-char and the character is returned.
Note that if it is not possible to complete the build of the character, the actions taken by read-char-no-hang depend on the situation:
nil is returned. (similar
means that the buffer may have been read during this operation, but
that the pointers are set so that the next octet read will be the same
as the first octet read for this operation).
See peek-char for the ANSI description.
For a Gray stream, peek-char calls stream-peek-char. Otherwise: a read-char equivalent is done, followed by an unread-char.
See listen for the ANSI description.
An extra argument above and beyond the CL spec is added to listen that specifies whether to listen for only an octet or to listen for a complete character. This solves the problem of having only a partial character in the stream, thus causing the next read-char to hang. The default for this extra listen-for-character argument is false, which provides the standard CL functionality to listen for any data.
For a Gray stream, listen calls stream-listen.
Otherwise: If the buffer is not empty, true is returned. Otherwise
device-read is called with a
null blocking argument. If that returns 0, then nil is returned, otherwise true is returned.
If the added optional argument is nil, only
an octet (8-bit byte) is looked for, otherwise external-format
processing is used to attempt to build a character in a non-blocking
way; if it is determined that the character can definitely be built,
then t is returned. However, the state of the stream is left in such a
way that an unread-char can be
done even after the listen (as is appropriate).
See read-line for the ANSI description.
For a Gray stream, read-line calls stream-read-line and processes the return values according to eof-error-p processing.
Otherwise: String buffers are allocated as necessary and read-char equivalent is performed until either a #\newline or eof is seen. A new string is allocated of the proper length and filled with the copied data from the temporary buffer(s) and then returned along with the missing newline flag.
Note: The read-line functionality can be optimized in the following way: A string buffer is allocated (this first one presumably on the stack) and read-char equivalent is performed until the next #\newline or eof is seen (or until the buffer is full, at which time new buffers are allocated as necessary). A new string of the proper length is then constructed and filled with the copied data from the temporary buffer(s) and then returned along with the missing newline flag.
Arguments: sequence stream &key start end partial-fill
See read-sequence for the ANSI description. Note that Allegro CL uses the additional partical-file keyword argument, which is not specified in ANSI CL.
For a Gray stream, read-sequence calls stream-read-sequence.
Otherwise: If the sequence is a string, then for every element of the string, a read-char equivalent is performed. Following the last read-character, the unread-char length is set (instead of at every character read).
If the sequence is an octet vector (i.e. a vector of (signed-byte 8) or (unsigned-byte 8) elements), then the equivalent of read-vector is performed.
Any other sequence type generates an error (for a simple-stream).
This argument controls the behavior when there are not enough objects
(of whatever is being read) on stream to fill the
sequence passed as the first argument (at least as far as
end, if given) and no EOF is seen. The ANSI
specification for read-sequence requires it to block is until
the sequence is filled or an EOF is seen. In the Allegro CL
implementation, the ANSI behavior (blocking) is observed if
partial-fill is nil
(the default).
If partial-fill is true, however, read-sequence will block for the first element, but will not block for any elements after the first, and so may return prior to the request being completed.
In all cases, read-sequence returns the index in the sequence of the next element not read.
Note: the partial-fill is new in release 6.0. In release 5.0.1, the non-blocking (and thus non-ANSI-compliant) behavior was implemented for buffered bivalent streams used for sockets in most but not all cases (the exact details are complicated). Users used to the non-blocking behavior will be surprised by the blocking behavior. Now, non-blocking behavior only occurs when partial-fill is specified true. Users expecting non-blocking behavior should specify partial-fill true.
See clear-input for the ANSI description.
For a Gray stream, clear-input calls stream-clear-input. Otherwise: if there is any input buffering in the stream, it is thrown away. Then device-clear-input is called.
See write-byte for the ANSI description.
For a Gray stream, write-byte calls stream-write-byte.
Otherwise: If the buffer is full, device-write is called to first write the buffer out, so that the buffer is made empty. An octet (8-bit byte) is expected as input. It is now stored into the stream's (non-full) buffer.
This is the lowest level functionality in the output portion of the CL API functions. Higher level functions which may call this function are: write-char, write-sequence, write-vector. Whether or not these functions actually call write-byte, call an internal but similar function, or expand all of write-byte's functionality inline is not specified.
See write-char for the ANSI description.
For a Gray stream, write-char calls stream-write-char.
Otherwise: If the character to be output is a control character, the
control-out table is consulted for a control-out function for that
character. If one exists it is assumed to be a function of two
arguments (the stream and the character), and is called for
device-level processing. If the control-out function exists and
returns non-nil, then no further action is taken for this character
since it was handled successfully in the control-out function. If the
control-out function does not exist or exists and returns nil, then normal processing continues for that
character. Normal processing means that the character is treated as
itself, to be sent uninterpreted to the stream.
The external-format functionality currently in effect is called for the character, which may result in any number of octets (8-bit bytes) being generated. These octets are then treated as if write-byte were called for each one, in the order they were received from the external-format processing.
See write-string for the ANSI description.
For a Gray stream, write-string calls stream-write-string.
Otherwise: For each character in the specified range in the string, the equivalent of a write-char is performed.
See write-sequence for the ANSI description.
For a Gray stream, write-sequence calls stream-write-sequence.
Otherwise: If the sequence is a string, then the equivalent of write-string is performed.
If the sequence is an octet vector (i.e. a vector of (signed-byte 8) or (unsigned-byte 8) elements), then the equivalent of write-vector is performed. Any other sequence type generates an error (for a simple-stream).
See terpri for the ANSI description.
For a Gray stream, terpri calls stream-terpri. Otherwise: the equivalent of a write-char of #\newline is performed.
See fresh-line for the ANSI description.
For a Gray stream, fresh-line
calls stream-fresh-line. Otherwise: if the stream
can be determined to be at the start of a line, then nothing is done
and nil is returned, otherwise the equivalent
of a write-char of #\newline
is performed.
See finish-output for the ANSI description.
For a Gray stream, finish-output calls
excl::stream-finish-output. Otherwise: if
there is any output in the stream's output buffer, is is written via
device-write with a non-nil
blocking argument.
Note that since Allegro CL does not queue writes, and since device-write calls are not required to write all of the requested bytes, the current implementation of finish-output loops on device-write calls until all of the unprocessed data are transferred.
See force-output for the ANSI description.
For a Gray stream, force-output calls stream-force-output. Otherwise: if there is
any output in the stream's output buffer, is is written via device-write with a
blocking argument of nil.
Note that since Allegro CL does not queue writes, and since device-write calls are not required to write all of the requested bytes, the current implementation of force-output is similar to finish-output, in that it loops on device-write calls until all of the unprocessed data are transferred.
See clear-output for the ANSI description.
For a Gray stream, clear-output calls stream-clear-output. Otherwise: the stream's output buffer is cleared and device-clear-output is called on the stream.
See file-position for the ANSI description.
For a Gray stream, file-position calls
excl::stream-file-position.
Otherwise: For simple-streams that are not string simple-streams, file-positions are always specified as a number of octets (8-bit bytes). For string simple-streams, file-positions are specified as number of characters.
If the position-spec argument is not given, the file position is calculated, possibly involving a call to device-file-position, and returned. Note that the file position may be precached in the stream, and device-file-position may have been called by some other CL functions.
If the position-spec argument is given, then the new file position is calculated and stored. This may involve a call to (setf device-file-position), if the position is outside of the buffer range.
See stream-element-type for the ANSI description.
For a Gray stream, stream-element-type returns the appropriate value.
For a simple-stream, stream-element-type always
returns (unsigned-byte 8).
These additional functions are provided in Allegro CL for operating on streams. Each is described on its own documentation page.
The endian-swap keyword argument to read-vector and write-vector allows the byte-ordering to be
controlled so as to allow big-endian and little-endian machines to
communicate with each other. Each version of Allegro CL has either
:big-endian or :little-endian on
its *features* list to identify it
appropriately. The endian-swap argument is effective only in
reads into and writes from vectors that are not strings, and is
silently ignored if given when a string is being passed to read-vector or
write-vector.
There are three kinds of value that can be given to the endian-swap argument:
:byte-8, :byte-16,
:byte-32, :byte-64, or
:byte-128 to indicate the width of the element
whose bytes to
reverse. For example, :byte-16 swaps every pair of
bytes,
and :byte-32 swaps every group of 4 bytes.
Note that :byte-8 does nothing to the byte-ordering,
and is included only
for symmetry.:network-order. This value does nothing on
big-endian machines, and on little-endian machines, causes bytes to be
swapped based on
the element-size of the vector being read into or written from.
For example, a
double-float vector, which has an element width of 64 bits,
is swapped on a :byte-64 basis on a little-endian
machine.The byte-swapping mechanism relies on the fact that objects are always aligned on 8 or 16 byte boundaries, depending on whether the lisp is a 32-bit or 64-bit lisp. Therefore, it is unadvisable to specify a numeric value of greater than 7 (in a 32-bit lisp) or 15 (in a 64-bit lisp).
The byte swapping mechanism is in fact implemented by performing a
logxor on the current index of the next byte to get
out of the vector. The resultant xor'ed index is used
as the true byte index into the array. No attempt is made to ensure
that the index is valid (within range): it is the user's
responsibility to ensure that. This is always ensured if the
endian-swap specification matches the element width of the vector
(e.g. an (unsigned-byte 16) vector is
given an endian-swap value of :byte-16,
or a double-float vector is given an endian-swap
value of :byte-64).
The following table shows how the bytes actually appear after swapping. Since the swapping is symmetrical, it can be used in either direction, for both reading and writing. Given the natural byte order of bytes A, B, C, D, E, F, G, H to start, the table shows the byte order of the resultant bytes for some example cases:
name value order
:byte-8 0 A B C D E F G H
:byte-16 1 B A D C F E H G
---- 2 C D A B G H E F
:byte-32 3 D C B A H G F E
---- 4 E F G H A B C D
---- 5 F E H G B A D C
---- 6 G H E F C D A B
:byte-64 7 H G F E D C B A
...
These functions are all written as if they call lower level CL functions, and do not necessarily call device-level functionality directly.
format
pprint
prin1
prin1-to-string
princ
princ-to-string
print
read
read-delimited-list
read-from-string
read-preserving-whitespace
write-line
write-to-string
The class hierarchy for streams starts with stream
at the head, and implementations which include other stream classes
such as Gray streams will place those stream classes as subclasses of
stream. In Allegro CL, the only subclasses of
stream are simple-stream and
fundamental-stream.
fundamental-stream denotes a Gray stream.
The simple-stream class hierarchy is divided into
three fundamental simple-stream classes (which in turn have subclasses
not listed in the diagram), based on the kinds of buffering they do:
--> fundamental-stream ...
| (Gray streams)
|
stream --+
| --> single-channel-simple-stream ...
| |
--> simple-stream --+--> dual-channel-simple-stream ...
|
--> string-simple-stream ...
These simple-stream subclasses cannot be mixed. They are intended to implement three styles of input/output in fundamentally different ways.
single-channel-simple-streamdual-channel-simple-streamstring-simple-stream
The basic behavior of the Common Lisp functions is described in 5.1 Implementation of Common Lisp Functions for Simple-Streams. This description must be taken on an as-if basis, which means that the specific functions described may not actually be called at all, or else they might be implemented using compiler-macros to call lower-level functions after type inferencing proofs have been established. However, the device-level interface does not have this freedom; those methods applicable for the stream class must be called in the way specified. This is to guarantee to the device-writer that methods that are written for a particular purpose will indeed be called.
However, the selection of methods to call when appropriate depends on the strategy used. Listed below are various sets of functions that are called for various stream types.
Whenever a control-character is seen when reading or writing on a stream, a decision must be made as to what to do with these characters. In a "raw" environment, the characters are processed as themselves; when writing they are inserted into the buffer (possibly after translation to octet form) and when reading they are simply returned as Lisp characters (possibly after having been assembled from octet form). In a "cooked" environment, at least some control characters turn into instructions at the device level, and are not inserted into or extracted from the stream as characters.
An example of this is terpri, which is simply a write-char of a #\newline. On a terminal stream, a terpri simply sends the #\newline as a character (though its sending may require a column indicator in the stream to be set to 0 as well). However, a window stream should not see a #\newline at the device-level, instead the action should be to "move the cursor down one line and to the far left side of the window".
The simple-streams design allows for both of these kinds of action. Each stream has two slots, a control-in slot and a control-out slot, which may contain tables of functions that are consulted when the character being read or written is determined to be a control character. The actions taken are as follows:
- If the control-out table has a function entry for the control-character being written, that function is called with two arguments: the stream and the character. The control-out function should perform whatever work that it is required to do, and return non-nil, meaning that it is finished processing that character, or else
nil, which means that the normal character processing action is taken which inserts the character into the stream.- If the control-in table has a function entry for the control-character that has just been read out of the stream, that function is called with two arguments: the stream and the character. The required actions are taken, and the function returns a new character to substitute (or the old character), or it may return
nilto indicate end-of-file. Note that the control-in handler must not try to do any reading from the stream at all; the intention for the control-in handler is to translate an already-received character to another, or to perform an operation and return a character. For ligatures and other multiple-character inputs, a composing external-format should be used or created, or else an encapsulation created for such translations.
Control-tables are built with make-control-table and are stored into the appropriate control-in or control-out slots by device-open.
This section gives some tips for device-writing. It is not comprehensive, and some of the functions and macros it refers to may or may not be documented. The section is Allegro CL specific, but may be taken as a guide for other implementations as well.
New stream classes may be created which subclass existing classes. If the superclass chosen is a currently instantiable class, such as terminal-simple-stream, file-simple-stream, etc., then the device methods may be used as they are, or they may be called by call-next-method by the more specialized method. If the superclass chosen is one of the three major streams (single-channel-simple-stream, dual-channel-simple-stream, or string-simple-stream) then much of the device functionality will have to be written from scratch. There may be some methods that exist to provide defaults (for example, the default device-buffer-length method specializes on simple-stream to provide a default for all simple-streams). Other methods, such as device-open, have no appropriate default action, and are thus not supplied.
To define a new stream class in Allegro CL, the iodefs module must be required to provide some defining macros. The class may be then defined using excl::def-stream-class:
(require :iodefs)
(def-stream-class blarg (terminal-simple-stream)
((slot1 :initform nil)
(slot2 :initform nil :accessor blarg-slot1))
(:default-initargs :input-handle
(error "blarg stream must have a :input-handle arg")))
Each primary method to device-open returns a stream that is fully connected to its device; it can perform all operations intended on that device. When a primary method performs a call-next-method to do a device-open on a less-specific device, that functionality is complete when the call-next-method returns.
For example, suppose a whiz-bang is a type of file which has a header line associated with it, to be internalized and then ignored as data. The whiz-bang stream might be defined as
(def-stream-class whiz-bang
(file-simple-stream)
((header :initform nil :accessor whiz-bang-header)))
The device-open for whiz-bang might call the primary-method for the file, and then do its own work afterward:
(defmethod device-open ((stream whiz-bang) options)
(declare (ignore options))
(let ((success (call-next-method)))
(when success ;; read and internalize the header
(setf (whiz-bang-header stream) (read-line stream))
t)))
Note that:
nil, which indicates that the
device-open
failed. File operations may thus be performed on the stream. A device-open that does not call-next-method must perform the following:
The following functions allow device-read and device-write methods to be implemented. They should never be used at any level other than the device-level, because they do only minimal checking on their arguments.
Note that the supplied device-read and device-write functions do not generate errors themselves, but pass them back to the higher level for processing. This allows read-octets and write-octets to pass errors back as well, as the implementation of a higher level (encapsulating) device-read and device-write.
The following operators are named by symbols exported from the excl package. They are loaded with
(require :iodefs)
Copyright (c) 1998-2000, Franz Inc. Berkeley, CA., USA. All rights reserved. Created 2000.10.5.