\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#462: Add New [info ps] Ensemble for Subprocess Management}
\date{January 23, 2017}
\author{Fr\'{e}d\'{e}ric Bonnet}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#462}&\textbf{Add New [info ps] Ensemble for Subprocess Management}\\\hline
Author&

Fr\'{e}d\'{e}ric Bonnet \tipmail{fredericbonnet}{free.fr}
\\
Created&Monday, \(\text{23}^{\text{rd}}\) January 2017\\
Type&Project\\
Tcl Version&\(8.7\)\\
State&Draft\\
Vote&Pending\\
Version&\tipversion$Revision: 1.4 $\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposes to improve Tcl's handling of subprocesses created by the \textbf{exec} and \textbf{open} commands by adding a new \textbf{::tcl::process} ensemble.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Rationale}

This TIP is inspired by a \url{https://github.com/flightaware/Tcl-bounties\#stop-tcl-from-eating-child-process-exit-status-gratuitously\%}\tipbar{}\%request from FlightAware\%\tipbar{}\% to fix the way Tcl currently handles child process exit status. 

Subprocess creation can be either synchronous or asynchronous. In either case, a children with a non-zero return value indicates an error condition that is bubbled up to the Tcl error handling mechanism. 

\subsection{Synchronous subprocesses}

Synchronous subprocesses are created using the \textbf{exec} command with no \textbf{\&} terminal argument. Errors are raised synchronously as well.

\subsection{Asynchronous subprocesses}

Asynchronous subprocesses can be created using two distinct methods:
\begin{itemize}
\item{}
\textbf{exec} command with a \textbf{\&} terminal argument. In this case the command returns immediately with a list of the PIDs for all the subprocesses in the pipeline.
\item{}
\textbf{open ``\tipbar{} command''}. In this case the command returns immediately with the channel id of the pipe (hereafter \textbf{\$ch}). The subprocess IDs are given by the \textbf{[pid \$ch]} command. The subprocess status code and error conditions are processed upon channel closure with the \textbf{[close \$ch]}.
\end{itemize}

\subsection{Error handling and status code}

Errors are caught with the \textbf{catch} and \textbf{try} commands, with status codes given in the \textbf{-errorcode} options dictionary entry and the \textbf{errorCode} global variable in the form \textbf{\{CHILDKILLED pid sigName msg\}} / \textbf{\{CHILDSTATUS pid code\}} / \textbf{\{CHILDSUSP pid sigName msg\}}. 

\subsection{C-level access}

The Tcl library provides the following procedures for managing subprocesses (excerpts from the Tcl documentation):
\begin{itemize}
\item{}
\textbf{Tcl\_DetachPids} may be called to ask Tcl to take responsibility for one or more processes whose process ids are contained in the pidPtr array passed as argument. The caller presumably has started these processes running in background and does not want to have to deal with them again.
\item{}
\textbf{Tcl\_ReapDetachedProcs} invokes the \textbf{waitpid} kernel call on each of the background processes so that its state can be cleaned up if it has exited. If the process has not exited yet, \textbf{Tcl\_ReapDetachedProcs} does not wait for it to exit; it will check again the next time it is invoked. Tcl automatically calls \textbf{Tcl\_ReapDetachedProcs} each time the exec command is executed, so in most cases it is not necessary for any code outside of Tcl to invoke Tcl\_ReapDetachedProcs. However, if you call \textbf{Tcl\_DetachPids} in situations where the exec command may never get executed, you may wish to call \textbf{Tcl\_ReapDetachedProcs} from time to time so that background processes can be cleaned up.
\item{}
\textbf{Tcl\_WaitPid} is a thin wrapper around the facilities provided by the operating system to wait on the end of a spawned process and to check a whether spawned process is still running. It is used by \textbf{Tcl\_ReapDetachedProcs} and the channel system to portably access the operating system.
\end{itemize}

Moreover, \textbf{Tcl\_WaitPid} is blocking unless called with the \textbf{WNOHANG} option.

\subsection{Limitations}

The current implementation is lacking several key features:
\begin{itemize}
\item{}
There is no way to get subprocess status other than through the error handling mechanism.
\item{}
Consequently, there is no way to collect the status code of a asychronous subprocess created with the \textbf{exec \&} method because such commands don't raise errors once the subprocesses are launched.
\item{}
There is no non-blocking way to query asynchronous subprocess status codes; \textbf{catch}/\textbf{try} upon \textbf{open \tipbar{}} pipe closure is blocking.
\item{}
Moreover, \textbf{exec} and \textbf{open} call \textbf{Tcl\_ReapDetachedProcs}, thereby cleaning up all pending information on terminated subprocesses. This prevents any advanced subprocess monitoring at the script level.
\item{}
While reasonable in the general case, a non-zero return value does not always indicates an error condition for all kinds of programs, so it is desirable to provide a subprocess-specific mechanism that does not rely on Tcl's standard error handling facility.
\end{itemize}

\section{Specifications}

A new \textbf{::tcl::process} will be created:

 \textbf{::tcl::process} \textit{subcommand ?arg ...}: Subprocess management.

The following \textit{subcommand} values are supported by \textbf{::tcl::process}: 
\begin{itemize}
\item{}
\textbf{::tcl::process list}: Returns the list of subprocess PIDs.
\item{}
\textbf{::tcl::process status} \textit{?switches? ?pids?}: Returns a dictionary mapping subprocess PIDs to their respective statuses. If \textit{pids} is specified as a list of PIDs then the command only returns the status of the matching subprocesses if they exist, and raises an error otherwise. The status value uses the same format as the \textbf{errorCode} global variable for terminated processes; for active processes an empty value is returned. Under the hood this command calls \textbf{Tcl\_WaitPid} with the \textbf{WNOHANG} flag set for non-blocking behavior. 
\item{}
\textbf{::tcl::process purge} \textit{?pids?}: Cleans up all data associated with terminated subprocesses. If \textit{pids} is specified as a list of PIDs then the command only cleanup data for the matching subprocesses if they exist, and raises an error otherwise. If the process is still active then it does nothing.
\item{}
\textbf{::tcl::process autopurge} \textit{?flag?}: Automatic purge facility. If \textit{flag} is specified as a boolean value then it activates or deactivate autopurge. In all cases it returns the current status as a boolean value. When autopurge is active, \textbf{Tcl\_ReapDetachedProcs} is called each time the \textbf{exec} command is executed or a pipe channel created by \textbf{open} is closed. When autopurge is inactive, \textbf{::tcl::process purge} must be called explicitly. By default autopurge is active and replicates the current Tcl behavior.
\end{itemize}

Additionally, \textbf{::tcl::process status} accepts the following switches:
\begin{description}
\item[* '''-wait''']
By default the command returns immediately (the underlying \textbf{Tcl\_WaitPid} is called with the \textbf{WNOHANG} flag set) unless this switch is set. If \textit{pids} is specified as a list of PIDs then the command waits until the matching subprocess statuses are available. If \textit{pids} is not specified then it waits for all known subprocesses.
\item[* '''-{}-''']
Marks the end of switches. The argument following this one will be treated as the first arg even if it starts with a -. 
\end{description}

\section{Examples}
\begingroup\small\begin{verbatim}
% ::tcl::process autopurge
true
% ::tcl::process autopurge false
false

% set pid1 [exec command1 a b c | command2 d e f &]
123 456
% set chan [open "|command1 a b c | command2 d e f"]
file123
% set pid2 [pid $chan]
789 1011

% ::tcl::process list
123 456 789 1011

% ::tcl::process status
123 {CHILDSTATUS 123 0} 456 {CHILDKILLED 456 SIGPIPE "write on pipe with no readers"} 789 {CHILDSUSP 789 SIGTTIN "background tty read"} 1011 {}

% ::tcl::process status 123
123 {CHILDSTATUS 123 0}

% ::tcl::process status 1011
1011 {}

% ::tcl::process status -wait
123 {CHILDSTATUS 123 0} 456 {CHILDKILLED 456 SIGPIPE "write on pipe with no readers"} 789 {CHILDSUSP 789 SIGTTIN "background tty read"} 1011 {CHILDSTATUS 1011 -1}

% ::tcl::process status 1011
1011 {CHILDSTATUS 1011 -1}

% ::tcl::process purge
% exec command1 1 2 3 &
1213
% ::tcl::process list
1213
\end{verbatim}
\endgroup

\section{Rejected Alternatives}

The first version proposed to implement the feature as a new \textbf{ps} option to the existing \textbf{info} command. However, almost all operations in \textbf{[info]} are things that just examine state, not change it, and that's a principle-of-least-astonishment that should be upheld for the sake of less experienced users.

\section{Reference implementation}

TBD

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\end{document}
