run-shell-command

Function

Package: excl

Arguments: command &key input output error-output separate-streams wait if-input-does-not-exist if-output-exists if-error-output-exists show-windowk

This function, originally written and named for Unix machines, is badly misnamed for Windows, because on Windows, you can only run programs. Shell commands, such as dir, are not acceptable as a value of the command argument. See below in this entry for discussion on this point.

Also note that there is often a Lisp tool equivalent to a command. (For example, the Lisp function directory can be used in place of dir on Windows and ls on Unix. See below under the heading run-shell-command and Windows for why "dir" cannot be the value of the command argument.) It is always preferable to use Lisp tools where possible.

This function returns either one or three or four values, depending on the value of wait and separate-streams. If wait is t, the single value returned is the exit status of the spawned process. If wait is nil, the three values returned are

stream-a stream-b process-id

where stream-a is a stream if either input or output is :stream and nil otherwise; and stream-b is a stream if error-output is :stream and nil otherwise. The process-id is the spawned process's process identification number. The user can communicate with the spawned process. Any shell command can be run in that process and its output can easily be communicated to Lisp. Writing to stream-a will send input to the spawned process if input is :stream, and reading from stream-a will read the output of the process if output is :stream.

The command argument can be a string containing a shell command (in Unix) or a program (in Windows). On Unix only, command can be a simple vector (element type t).

If command is a string, a shell is spawned which parses and executes the command.
If command is a simple vector (Unix only), its elements should be coercible to strings. The zeroth element and the remainder of the vector, converted to C strings, are passed as the two arguments to execvp() (and thus bypassing the intermediate shell process despite the name of this function). This is faster than spawning a shell and it avoids processing of shell initialization files such as .cshrc.

Here are two Unix invocations, one using a vector and one a string:

(run-shell-command #("ls" "ls" "-l" "dcl"))
(run-shell-command "ls -l dcl")

Note that when command is a string, and wait is nil, the returned process-id will be that of the shell, not the command itself, unless the command is prefaced with the shell `exec' builtin command. Thus

(run-shell-command "ls" :wait nil)

will return the process-id of the shell in which ls is run while

(run-shell-command "exec ls" :wait nil)

will return the process-id of ls as will

(run-shell-command #("ls" "ls") :wait nil)

The values of input, output, and error-output control what the process will use as standard input (file descriptor 0), standard output (file descriptor 1) and standard error (file descriptor 2). The values can be

nil - inherit standard input, output or error (respectively) from Lisp. Note that this is Unix standard input and output inherited from the Lisp process and is unrelated to the Lisp variables *standard-input* and *standard-output*.
a pathname (or string) - open the file specified by the pathname (or string) and use the resulting stream;
a stream - use the stream. The stream must be of a kind that has a Unix file descriptor in the appropriate direction. Examples include a stream open to a file or a hardware device or a Unix socket. A string output stream, for example, is not an appropriate value;
:stream - create a stream and return it to the Lisp program. Since waiting and having a stream open to a process can cause the process to hang, :wait must be nil; if :wait is t, an error will result.

error-output has an additional allowed value: :output, which directs standard error to the same place as standard output.

The default value in all cases is nil.

wait may be t or nil. If :wait is t, Lisp will wait for the command to exit before resuming. If wait is nil, Lisp will start the process and then resume without waiting for it to finish. The default for wait is t.

Note that if wait is specified as nil, then the process will remain in the system after it completes until either Lisp exits or Lisp executes reap-os-subprocess to inquire about the exit status. To prevent the system becoming clogged with processes, a program that spawns a number of processes with :wait set to nil must be sure to call reap-os-subprocess after each process finishes. Further, the stream returned by run-shell-command must be closed (the system will not close them automatically).

The keywords if-input-does-not-exist, if-output-exists and if-error-output-exists are used when input, output or error-output are pathnames (or strings).

Lisp uses open to open a stream to the file identified by the pathname (or string) and the values of if-input-does-not-exist, if-output-exists and if-error-output-exists are passed to the open function as the value of its if-does-not-exist parameter (for input) and the if-exists parameter (for output and error-output.)

The permissible values for if-does-not-exist are :error, :create and nil. Those for if-output-exists and if-error-output-exists are :error, :overwrite, :append, :supersede and nil. The default is :error in all cases.

The separate-streams keyword argument causes separate streams to be returned for input and output. In all cases, nil will be returned in place of a stream if the run-shell-command arguments don't ask that a particular stream be created. The value of separate-streams only matters when the value of the wait is nil. If separate-streams is true, four rather than three values are returned, the first two being the input and output streams, then the error stream and the process id.

The show-window keyword argument only has effect on Windows. The value controls how the window created by the program run by run-shell-command first appears. The value can be an integer. The integer should be the value of one of the SW_ constants defined in the winuser.h include file that is part of the Windows SDK. The symbolic value should be preferred over an integer, however, since this will be portable in the face of changes to the Windows SDK header files and use on other operating systems. The value can also be one of the following symbols:

`nil`	Let the process itself determine how it will show its window.
`:normal`	Display a window in neither a maximized nor minimized state. Same as integer value SW_NORMAL.
`:showna`	Show but do not select. Same as integer value SW_SHOWNA.
`:hide`	Hide the window completely. It doesn't even appear in the task bar. Same as integer value SW_HIDE.
`:minimized`	Display the window as an icon in the task bar. Same as integer value SW_MINIMIZED.
`:maximized`	Display the window as a full screen maximized window. Same as integer value SW_MAXIMIZED. Console windows can not be maximized, however.

run-shell-command and Windows

On Windows 95 and NT, run-shell-command executes programs but does not invoke shell commands. The function is therefore misnamed for Windows. It is called run-shell-command to provide cross-platform compatibility between Windows and Unix and back compatibility for ACL4.3.x users. It does behave differently on Windows and Unix. The difference is related to differences between UNIX and Windows.

On Unix run-shell-command spawns a Unix process and runs the executable program or Unix shell command provided as argument to the function. On Windows, the argument to run-shell-command command is started directly and so works with programs but the Windows command shell is not invoked and so run-shell-command does not work with shell commands. An example of a program is Notepad. Either of these forms will start Notepad on Windows 95 and NT:

(run-shell-command "notepad") 
(run-shell-command "notepad.exe")

To run a DOS shell command, the argument to run-shell-command has to start the shell and tell it to run the command. On Windows 95 the name of the shell is command.com so this form will run a shell command on Windows 95:

(run-shell-command "start command /k dir")

A more complete example (where the results are read into Lisp) is:

(let ((str (run-shell-command "start command /k dir" :wait nil :output :stream))) 
  (do ((ch (read-char str nil nil) (read-char str nil nil))) 
      ((null ch) (close str) (sys:os-wait)) (write-char ch)))

On Windows NT the name of the shell is cmd.exe but it is possible to run command.com. An example of using run-shell-command on NT 4.0:

(run-shell-command "cmd /c start cmd /k dir")

Running shell commands on Windows 95 works less well than doing so in NT, possibly because the Windows 95 shell is older technology than the shell in NT.

Also:

The first argument to run-shell-command has to be a string (the Unix version will accept a vector of strings).
The run-shell-command command will be started directly and the Windows command shell will not be invoked to start the command. Because the Windows shell is not involved, you cannot include shell operators in the command (e.g.<<, <, >, and | ).
If the keyword arguments input, output and error-output are given stream values, then that stream must be open to a file on the disk. In particular a stream such as *terminal-io* will not be accepted here, nor will a socket stream nor a stream returned by a previous run-shell-command.
You can start a DOS (Console mode) or a Windows program with run-shell-command. Note: Windows programs are unlikely to use standard input, output or error.
When you use streams to communicate with a spawned process, keep in mind that the streams are operating in a raw mode that doesn't translate a carriage-return linefeed to a linefeed on input, and a linefeed to a carriage return followed by a linefeed on output.

run-shell-command and Unix

run-shell-command starts a process with execlp() when command is a string and (as we said above) execvp() when command is a simple vector.

The SHELL environment variable is used to determine the type of shell spawned. If SHELL has no value. /bin/csh is used. If that fails, /bin/sh is used.

The environment in which run-shell-command runs its processes. run-shell-command starts up subprocesses of the Lisp process. In order to modify an environment variable for these subprocesses, you must modify the environment for the Lisp process -- it does no good to modify the environment for the parent process of the Lisp process (i.e. the shell where Lisp was started, or Emacs, if Lisp is started as a subprocess of Emacs).

You can poll environment variables with getenv, and you can set environment variables for the Lisp process, and thus for subprocesses created by Lisp (by run-shell-command, e.g.) with setf and getenv.

A couple of Unix examples

;; The current values of the environment variables on your system may,
;; of course, be different from what appears in this example.
user(2): (sys:getenv "SHELL")
"/bin/csh"
user(3): (setf (sys:getenv "SHELL") "/bin/sh")
"/bin/sh"
user(4): (sys:getenv "SHELL")
"/bin/sh"

Here are a couple of examples of run-shell-command. In the first, we simply have run-shell-command execute a simple command (who).

USER(1): (run-shell-command "who")
rlogin ttyb Aug 19 08:26
sdj ttyp0 Aug 18 16:08 (rubix)
adam ttyp2 Aug 18 16:17 (rubix)
dm ttyp4 Aug 19 10:24 (rubix)
0
USER(2):

The second example is more complicated. We cause run-shell-command to spawn a shell and then we communicate with the shell via the stream set up by run-shell-command.

;; First we define a function to read the output from the shell. This
;; function is pretty simple -- it reads characters and prints them
;; out but it does show how a more useful function could be implemented.
USER(24): (defun get-from-shell (stream)
 (do ((ch
(read-char-no-hang stream)

(read-char-no-hang stream)))

((null ch))

(write-char ch)))
GET-FROM-SHELL
;; Now we initiate the shell:
USER(25): (setq shell-stream (excl:run-shell-command "csh"

:input :stream

:output :stream

:wait nil))
#<EXCL::BIDIRECTIONAL-TERMINAL-STREAM @ #x10a4aa6>
USER(26): (format shell-stream "who~%")
NIL
USER(27): (force-output shell-stream)
NIL
USER(28): (get-from-shell shell-stream)
rlogin ttya Aug 19 07:06
rlogin ttyb Aug 19 08:26
sdj ttyp0 Aug 18 16:08 (rubix)
cheetham ttyp1 Aug 18 17:17 (frozen)
adam ttyp2 Aug 18 16:17 (rubix)
NIL
;; We exit the shell:
USER(29): (format shell-stream "exit~%")
NIL
;; and close the stream.
USER(30): (close shell-stream)
T
;; We call sys:reap-os-subprocess because we called run-shell-command with :wait nil:
USER(31): (sys:reap-os-subprocess)
0
3995
USER(32):

excl:run-shell-command and multiprocessing

run-shell-command does not take multiprocessing into consideration. Therefore, if it is called with the :wait argument true (the default is t), all of Lisp waits for the call to complete, not just the process or thread that called run-shell-command. It is that behavior which is multiprocessing unfriendly. The following is a multiprocessing friendly call to run-shell-command. It does cause the calling process or thread to wait but does not cause the entire Lisp process to wait for the shell command to finish.

(multiple-value-bind (s errs my-pid)
    (run-shell-command "sleep 5; ls /usr/bin" :wait nil)
  (declare (ignore errs s))
  (let ((my-status nil))
    (mp::process-wait "for run-shell-command to finish"
                      #'(lambda ()
                         (setq my-status
                          (or my-status
                             (sys:reap-os-subprocess
                              :pid my-pid :wait nil)))))
    my-status))

Notes:

Calling run-shell-command with :wait nil allows Lisp to continue in any case, and that might be what you want.
To repeat what we said above, most things can be done better from Lisp without recourse to run-shell-command, which is inherently risky (to some extent, it puts the fate of the Lisp process in the hands of a non-Lisp program over which Lisp may not have control). Whenever you are tempted to use run-shell-command, consider performing the same action within Lisp (perhaps using foreign functions).

See also os-interface.htm for general information on the interface between in Allegro CL and the operating system and for information on shell commands.

The documentation is described in introduction.htm and the index is in index.htm.

Created 2000.10.5.