:TITLE: Invoking Subprocesses from Tcl - exec, open
;#
;# RCSID: $Header: /cvsroot/tcl/tcltutorial/original/Tcl26.lsn,v 1.1 2004/11/04 16:01:14 davidw Exp $
;# Copyright (c) 1995 Clif Flynt
;# 9300 Fleming Rd.
;# Dexter, MI 48130
;# clif@cflynt.com
;# See file "NOTICE" for licensing terms.
;#
::CMD:: if {([info exists tcl_platform])} {
switch $tcl_platform(platform) {
unix {
set Tutor(lsn.codeMod) {
{regsub -nocase wish [info nameofexecutable] tclsh largv0 }
{regsub TCL_INTERP $line $largv0 line}
{regsub TEMPDIR $line "/tmp" line}
{regsub "wish" $line "tclsh" line}
}
}
windows {
set Tutor(lsn.codeMod) {
{regsub -nocase wish [info nameofexecutable] tclsh largv0 }
{regsub TCL_INTERP $line [file attributes $largv0 -shortname] line}
{regsub TEMPDIR $line "C:/temp" line}
{regsub "wish" $line "tclsh" line}
}
}
mac {
set Tutor(lsn.codeMod) {
{regsub -nocase wish [info nameofexecutable] tclsh largv0 }
{regsub TCL_INTERP $line $largv0 line}
{regsub "wish" $line "tclsh" line}
}
}
default {
puts "I don't recognize the platform: $tcl_platform(platform)"
puts "Can't set platform specific parameters"
}
}
}
:LESSON_TEXT_START_LEVEL 0:
So far the lessons have dealt with programming within the Tcl interpreter.
However, Tcl is also useful as a scripting language to tie other packages
together.
To accomplish this function, Tcl has methods for invoking subprocesses.
There are two ways to invoke a subprocess from Tcl:
- open
...... run a new program with I/O connected to a file descriptor
- exec
...... run a new program as a subprocess
The open call is the same call that is used to open a file.
If the first character in the file name argument is a pipe symbol (|),
then open will treat the rest of the argument as a program name,
and will exec that program with the standard input or output
connected to a file descriptor. A pipe can be opened to a sub-process for
reading, writing or both reading and writing.
If the file is opened for both reading and writing you must be aware that
the pipes are buffered. The output from a puts command will be
saved in an I/O buffer until the buffer is full, or until you execute a
flush command to force it to be transmitted to the subprocess.
The output of the subprocess will not be available to a read or
gets until the I/O buffer for the subprocess has filled its
output buffer.
The exec call is similar to invoking a program ( or a set of
programs piped together) from the shell prompt or in a unix
shell script. It supports several styles of output redirection, or it
can return the output of the sub-process as the return of the exec
call.
- open |
progName ?access?
- Returns a file descriptor for the pipe. The progName
argument must start with the pipe symbol. If progName is
enclosed in quotes or braces, it can include arguments to the
subprocess.
- exec
?switches? arg1 ?arg2? ... ?argN?
- Exec treats its arguments as the names and arguments for
a set of subprocesses to execute. If the first args start
with a "-", then they are treated as switches to the
exec command, instead of being invoked as subprocesses or
subprocess options.
Switches are:
- -keepnewline
- Retains a trailing newline in the pipeline's output. Normally a
trailing newline will be deleted.
- --
- Marks the end of the switches. The next string will be treated
as arg1, even if it starts with a "-"
Arg1 - argN can be one of:
- the name of a program to execute
- an command line argument for the subprocess
- an I/O redirection instruction.
There are many permutations to the I/O redirection commands.
The main subset of these commands is:
- |
- Pipes the standard output of the command preceeding the pipe
symbol into the standard input of the command following the pipe symbol.
- < fileName
- The first program in the pipe will read input from fileName.
- < @ fileID
- The first program in the pipe will read input from the Tcl
descriptor fileID.
FileID is the value returned from an open ... "r" command.
- < < value
- The first program in the pipe will read value as its input.
- > fileName
- The output of the last program in the pipe will be sent to fileName.
Any previous contents of fileName will be lost.
- > > fileName
- The output of the last program in the pipe will be appended to fileName.
- 2> fileName
- The standard error from all the programs in the pipe will be sent to fileName.
Any previous contents of fileName will be lost.
- 2> > fileName
- The standard error from all the programs in the pipe will be appended to fileName.
- > @ fileID
- The output from the last program in the pipe will be written to fileID.
FileID is the value returned from an open ... "w" command.
:TEXT_END:
:LESSON_TEXT_START_LEVEL 1:
So far the lessons have dealt with programming within the Tcl interpreter.
However, Tcl is also useful as a scripting language to tie other packages
together.
To accomplish this function, Tcl has methods for invoking subprocesses.
There are two ways to invoke a subprocess from Tcl:
- open
...... run a new program with I/O connected to a file descriptor
- exec
...... run a new program as a subprocess
The open call is the same call that is used to open a file.
If the first character in the file name argument is a pipe symbol (|),
then open will treat the rest of the argument as a program name,
and will exec that program with the standard input or output
connected to a file descriptor. A pipe can be opened to a sub-process for
reading, writing or both reading and writing.
If the file is opened for both reading and writing you must be aware that
the pipes are buffered. The output from a puts command will be
saved in an I/O buffer until the buffer is full, or until you execute a
flush command to force it to be transmitted to the subprocess.
The output of the subprocess will not be available to a read or
gets until the I/O buffer for the subprocess has filled its
output buffer.
The exec call is similar to invoking a program ( or a set of
programs piped together) from the shell prompt or in a unix
shell script. It supports several styles of output redirection, or it
can return the output of the sub-process as the return of the exec
call.
- open |
progName ?access?
- Returns a file descriptor for the pipe. The progName
argument must start with the pipe symbol. If progName is
enclosed in quotes or braces, it can include arguments to the
subprocess.
- exec
?switches? arg1 ?arg2? ... ?argN?
- Exec treats its arguments as the names and arguments for
a set of subprocesses to execute. If the first args start
with a "-", then they are treated as switches to the
exec command, instead of being invoked as subprocesses or
subprocess options.
Switches are:
- -keepnewline
- Retains a trailing newline in the pipeline's output. Normally a
trailing newline will be deleted.
- --
- Marks the end of the switches. The next string will be treated
as arg1, even if it starts with a "-"
Arg1 - argN can be one of:
- the name of a program to execute
- an command line argument for the subprocess
- an I/O redirection instruction.
There are many permutations to the I/O redirection commands.
The main subset of these commands is:
- |
- Pipes the standard output of the command preceeding the pipe
symbol into the standard input of the command following the pipe symbol.
- < fileName
- The first program in the pipe will read input from fileName.
- < @ fileID
- The first program in the pipe will read input from the Tcl
descriptor fileID.
FileID is the value returned from an open ... "r" command.
- < < value
- The first program in the pipe will read value as its input.
- > fileName
- The output of the last program in the pipe will be sent to fileName.
Any previous contents of fileName will be lost.
- > > fileName
- The output of the last program in the pipe will be appended to fileName.
- 2> fileName
- The standard error from all the programs in the pipe will be sent to fileName.
Any previous contents of fileName will be lost.
- 2> > fileName
- The standard error from all the programs in the pipe will be appended to fileName.
- > @ fileID
- The output from the last program in the pipe will be written to fileID.
FileID is the value returned from an open ... "w" command.
If you are familiar with shell programming, there are a few differences to
be aware of when you are writing Tcl scripts that use the exec
and open calls.
- You don't need the quotes that you would put around arguments to
escape them from the shell expanding them. In the example, the
argument to sed is not put in quotes. If it were put in quotes, the
quotes would be passed to sed, instead of being stripped off (as the
shell does), and sed would report an error.
- If you use the open |cmd "r+" construct, you must follow
each puts with a flush to force Tcl to send the command from its
buffer to the program. The output from the subprocess may be buffered
in its output buffer.
You can sometimes force the output from the sub-process to flush by
sending an exit command to the process.
You can also use the fconfigure command to make a
channel unbuffered.
The expect extension to Tcl provides a much better interface
to other programs, which handles the buffering problem.
- If one of the commands in an open |cmd fails the open
does not return an error. However, attempting to read input from the
file descriptor with gets $file will return an empty string.
Using the gets $file input construct will return a character
count of -1.
Put quotes around the s/.Q//g in the example to see this behavior.
- If one of the commands in an exec call fails to execute,
the exec will return an error, and the error output will include
the last line describing the error.
:TEXT_END:
:LESSON_TEXT_START_LEVEL 2:
So far the lessons have dealt with programming within the Tcl interpreter.
However, Tcl is also useful as a scripting language to tie other packages
together. This is similar to
the behavior of the DOS .bat files. Like the .bat files, you can invoke
other programs from Tcl. The new program is called a subprocess.
There are two ways to invoke a subprocess from Tcl:
- open
...... run a new program with I/O connected to a file descriptor
- exec
...... run a new program as a subprocess
The open call is the same call that is used to open a file.
If the first character in the file name argument is a pipe symbol (|),
then open will treat the rest of the argument as a program name,
and will exec that program with the standard input or output
connected to a file descriptor. A pipe can be opened to a sub-process for
reading, writing or both reading and writing.
Many computer operating systems buffer their input and output. This means
that the string you send to a puts may not be transmitted immediately.
Instead it is written into a buffer, and when the buffer is full (or the
program executes a flush command) the output is actually sent.
When these programs are run interactively, they usually flush the buffer
immediately after each output, but when they are run in a pipeline between
other processes, they buffer and batch the output. This is more efficient
for the computer.
If the file is opened for both reading and writing you must be aware that
the pipes are buffered. The output from a puts command will be
saved in an I/O buffer until the buffer is full, or until you execute a
flush command to force it to be transmitted to the subprocess.
The output of the subprocess will not be available to a read or
gets until the I/O buffer for the subprocess has filled its
output buffer.
The exec call is similar to invoking a program ( or a set of
programs piped together) from the shell prompt or in a unix
shell script. It supports several styles of output redirection, or it
can return the output of the sub-process as the return of the exec
call.
- open |
progName ?access?
- Returns a file descriptor for the pipe. The progName
argument must start with the pipe symbol. If progName is
enclosed in quotes or braces, it can include arguments to the
subprocess.
- exec
?switches? arg1 ?arg2? ... ?argN?
- Exec treats its arguments as the names and arguments for
a set of subprocesses to execute. If the first args start
with a "-", then they are treated as switches to the
exec command, instead of being invoked as subprocesses or
subprocess options.
Switches are:
- -keepnewline
- Retains a trailing newline in the pipeline's output. Normally a
trailing newline will be deleted.
- --
- Marks the end of the switches. The next string will be treated
as arg1, even if it starts with a "-"
Arg1 - argN can be one of:
- the name of a program to execute
- an command line argument for the subprocess
- an I/O redirection instruction.
There are many permutations to the I/O redirection commands.
The main subset of these commands is:
- |
- Pipes the standard output of the command preceeding the pipe
symbol into the standard input of the command following the pipe symbol.
- < fileName
- The first program in the pipe will read input from fileName.
- < @ fileID
- The first program in the pipe will read input from the Tcl
descriptor fileID.
FileID is the value returned from an open ... "r" command.
- < < value
- The first program in the pipe will read value as its input.
- > fileName
- The output of the last program in the pipe will be sent to fileName.
Any previous contents of fileName will be lost.
- > > fileName
- The output of the last program in the pipe will be appended to fileName.
- 2> fileName
- The standard error from all the programs in the pipe will be sent to fileName.
Any previous contents of fileName will be lost.
- 2> > fileName
- The standard error from all the programs in the pipe will be appended to fileName.
- > @ fileID
- The output from the last program in the pipe will be written to fileID.
FileID is the value returned from an open ... "w" command.
There are a few points to
be aware of when you are writing Tcl scripts that use the exec
and open calls.
- If you are used to shell programming, or typing a command from the
shell prompt, you don't need the quotes that you would put around arguments to
escape them from the shell expanding them. In the example, the
argument to sed is not put in quotes. If it were put in quotes, the
quotes would be passed to sed, instead of being stripped off (as the
shell does), and sed would report an error.
- If you use the open |cmd "r+" construct, you must follow
each puts with a flush to force Tcl to send the command from its
buffer to the program. The output from the subprocess may be buffered
in its output buffer.
You can sometimes force the output from the sub-process to flush by
sending an exit command to the process.
You can also use the fconfigure command to make a
channel unbuffered.
The expect extension to Tcl provides a much better interface
to other programs, which handles the buffering problem.
- If one of the commands in an open |cmd fails the open
does not return an error. However, attempting to read input from the
file descriptor with gets $file will return an empty string.
Using the gets $file input construct will return a character
count of -1.
Make the This will come back backwards string just
th to see this behavior in the example.
- If one of the commands in an exec call fails to execute,
the exec will return an error, and the error output will include
the last line describing the error.
The example code shows how to use the open and exec calls
in a couple ways. These aren't necessarily the best way to achieve a result,
but they work. In particular, if you are processing a large amount of text,
(and are on a system that has text oriented tools)
it is frequently faster to exec a copy of sed and pipe the text through
there instead of writing a Tcl script to do the processing.
The example starts out by writing a simple Tcl program that will
reverse the order of characters to a temporary file. We'll be running
that program in a separate Tcl interpreter to test reading and writing
to another program.
Once the program is written, and the file is closed, we can execute
that Tcl script. The command:
set io [open "|/usr/local/bin/tclsh $tempFileName" r+]
opens a pipe for reading and writing to the new program. When a string
is written to the program, it must be followed by a flush
command to force the string to be sent.
When the Tcl script reads a string from stdin it
reverses the letters, writes it to stdout, and exits. When this program
exits, the buffer is flushed, and the example can read the string
from the channel.
The exec example shows a different method for invoking
our sample program. A single line of input can be sent to another
program with a single Tcl command.
:TEXT_END:
:CODE_START:
# Create a unique (mostly) file name for a Tcl program
set tempFileName "TEMPDIR/inv_[pid].tcl"
# Open the output file, and
# write a simple program to it
set outfl [open $tempFileName w]
puts $outfl {
set len [gets stdin line]
if {$len < 5} {exit -1}
for {set i $len} {$i >= 0} {incr i -1} {
append l2 [string range $line $i $i]
}
puts $l2
exit 0;
}
# Flush and close the file
flush $outfl
close $outfl
# Run the new Tcl file interactively
# Open a pipe to the program
set io [open "|TCL_INTERP $tempFileName" r+]
# send a string to the new program
# *MUST FLUSH*
puts $io "This will come back backwards."
flush $io
# Get the reply, and display it.
set len [gets $io line]
puts "To reverse: 'This will come back backwards.'"
puts "Reversed is: $line"
puts "The line is $len characters long"
# Run the program with input defined in an exec call
set invert [exec TCL_INTERP $tempFileName << \
"ABLE WAS I ERE I SAW ELBA"]
# display the results
puts "The inversion of 'ABLE WAS I ERE I SAW ELBA' is \n $invert"
# Clean up
file delete $tempFileName
:TEXT_END: