\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#387: Unified Yield Command Syntax}
\date{May 30, 2011}
\author{Lars Hellström}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#387}&\textbf{Unified Yield Command Syntax}\\\hline
Author&

Lars Hellström \tipmail{Lars.Hellstrom}{residenset.net}
\\
Created&Monday, \(\text{30}^{\text{th}}\) May 2011\\
Type&Project\\
Tcl Version&\(8.6\)\\
State&Withdrawn\\
Vote&Pending\\
Version&\tipversion$Revision: 1.7 $\\
Keywords&Tcl, coroutine\\
Obsoleted By&396\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This document describes the \textit{syntax} of a unified \textbf{yield} command, which is designed so that it can combine all features of the various yield-related commands proposed during 2010.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\subsection{Note}

This should not be taken as a proposal to necessarily provide all the mentioned features right now, but rather as a roadmap to make a command into which they \textit{could} all be fitted. The purpose is to avoid choosing a direction that would get really awkward in the future.

\section{General Considerations}

It is accepted as given that the present syntax of the \textbf{yield} command, namely

\begin{quote}
\textbf{yield} ?\textit{value}?
\end{quote}

should continue to be valid. It is also desirable to control all additional features through options placed before the \textit{value}, since that makes it possible to let an \textbf{interp alias} supply them.

What is important given those constraints is that one must be able to determine whether the final argument is a \textit{value} or not solely from looking at the previous arguments, since any option name is also a perfectly valid \textit{value} for \textbf{yield} to return to the coroutine-caller. This can be accomplished through the rule that every option must consist of at least two words (typically -name and value); a word after an option is then known to be the \textit{value} if it is the very last word of the command, and known to be the first word of another option if it not the very last word of the command. (By contrast, an attempt to get by with a \textbf{-{}-} end-of-options option would become very complicated and error-prone.)

The overall command syntax would thus be similar to that of \textbf{return} (options before an optional value), although there is no need to reproduce any of return's options in \textbf{yield}, as all such functionality can be attained by yielding to \textbf{return}.

\section{Possible Options}

We identify the following options: \textbf{-arguments}, \textbf{-delivery} and \textbf{-to}. These are described below:

\subsection{Argument Specifiers}

The option

\begin{quote}
\textbf{-arguments} \textit{argspec}
\end{quote}

would be used to specify a \textbf{proc}-style list of arguments for the coroutine-command. As mentioned in \cite{tip372}, \textbf{yieldm} can then be defined as:
\begingroup\small\begin{verbatim}
   interp alias {} yieldm {} yield -arguments args
\end{verbatim}
\endgroup

whereas the default for \textbf{-arguments} is \textbf{\{arg ""\}}, as:
\begingroup\small\begin{verbatim}
   yield -arguments {{arg ""}}
\end{verbatim}
\endgroup

explicitly says the coroutine-command should have one optional argument \textit{arg} with default value the empty string.

The idea is that if the arguments supplied to the coroutine-command do not match the \textit{argspec}, then the coroutine-command should throw an appropriate error rather than resuming the coroutine. In that case, the coroutine state does not change, and in particular the coroutine-command continues to use the same \textit{argspec}.

For the purpose of further discussion below, the elements in the \textit{argspec} list will be called \textbf{formal parameters} of the coroutine-command.

\subsection{Yielding To}

The \textbf{-to} option provides the functionality of the \textbf{yieldto} command, and has the syntax:

\begin{quote}
\textbf{-to} \textit{cmdname} ?\textit{arg ...}?
\end{quote}

always absorbing all remaining arguments of the \textbf{yield} command; a precedent for this may be found in for example the \textbf{-join} option of the \textbf{glob} command. In order to meet the ``at least two words'' requirement for \textbf{yield} options, the \textit{cmdname} argument is mandatory, but this is quite natural since it is anyway to be resolved already by \textbf{yield}.

Note that behavior like yielding and throwing an error can be performed by yielding to \textbf{return}:
\begingroup\small\begin{verbatim}
   yield -to return -code error -errorcode DEMO "Just an example"
\end{verbatim}
\endgroup

\textbf{-code} and \textbf{-errorcode} here are not options of \textbf{yield}, but of \textbf{return}, although the impression is probably close to that of having them options of \textbf{yield} directly.

\subsection{Argument Delivery}

An issue where no obvious solution presented itself was that of how arguments supplied to the coroutine-command should be made available in the coroutine. This proposal suggest that a \textbf{-delivery} option could be used to control this, and that the word after \textbf{-delivery} is a \textit{mode} keyword which selects whether the arguments should be provided as a list, a dictionary, in variables, or whatever. The following is a list of possible forms of this option.

The supported modes will be one of: \textbf{dict}, \textbf{first}, \textbf{flat}, \textbf{same} and \textbf{vars}. These are described below. (Note that different modes need not have the same number of argument words.)

\subsubsection{Dictionary Delivery}

\begin{quote}
\textbf{-delivery dict}
\end{quote}

The return value of \textbf{yield} is a dictionary, with one entry for each formal parameter, the entries appearing in the same order as in the \textbf{-arguments} \textit{argspec}, and the values filled in as they would be for a \textbf{proc} call. The latter implies that \textit{default values are filled in for missing optional arguments}; this preserves the invariant for the caller that specifying the default value for an optional argument is the same as omitting that argument.

Thus, for the demonstration code:
\begingroup\small\begin{verbatim}
 proc demo {delivery} {
    set last ""
    foreach argspec {
       foo {{arg ""}} {{arg ""}} args {foo args}
    } {
       set last [yield -arguments $argspec -delivery $delivery $last]
    }
    return $last
 }
\end{verbatim}
\endgroup

one can get this interactive session:
\begingroup\small\begin{verbatim}
 % coroutine C demo dict
 % C "a b"
 foo {a b}
 % C "a b"
 arg {a b}
 % C
 arg {}
 % C "a b"
 args {{a b}}
 % C "a b"
 foo {a b} args {}
\end{verbatim}
\endgroup

\subsubsection{Flat Delivery}

It is not unreasonable to also expect a mode that delivers just the list of formal parameter values, e.g.

\begin{quote}
\textbf{-delivery list}
\end{quote}

but this can be obtained from
\begingroup\small\begin{verbatim}
   dict values [yield -delivery dict ...]
\end{verbatim}
\endgroup

since the \textbf{dict} mode is specified as putting the entries in the right order.

Instead there is another take on returning a list of arguments that provides extra functionality:

\begin{quote}
\textbf{-delivery flat}
\end{quote}

The return value of \textbf{yield} is the list of arguments of the coroutine-command, regardless of \textbf{-arguments} \textit{argspec}. This is the \textbf{yield} counterpart of \textbf{info level 0}, providing full introspection into the arguments, but also leaving parsing entirely up to the programmer.

With the same sequence of calls as above, one gets
\begingroup\small\begin{verbatim}
 % coroutine C demo flat
 % C "a b"
 {a b}
 % C "a b"
 {a b}
 % C
 % C "a b"
 {a b}
 % C "a b"
 {a b}
\end{verbatim}
\endgroup

\subsubsection{Delivery to Local Variables}

\begin{quote}
\textbf{-delivery vars} \textit{varlist}
\end{quote}

The \textit{varlist} must be a list with one element for each formal parameter. The values of the formal parameters are assigned, in sequence, to the variables named in the \textit{varlist}. The return value is an empty string.

Of course, most of the time one would use the same names as in the \textit{argspec}, so it is reasonable to have a shorthand for this:

\begin{quote}
\textbf{-delivery same}
\end{quote}

The main reason to do otherwise is if both the procedure \textbf{yield}ing and the coroutine-command has an \textbf{args} argument, since that is a magical name. Hiding implementation details from users can also be a reason to specify a different \textit{varlist}.

A further possibility that has been suggested would be:

\begin{quote}
\textbf{-delivery array} \textit{arrname}
\end{quote}

for dumping the arguments as entries in an array. This can however be accomplished through
\begingroup\small\begin{verbatim}
 array set $arrname [yield -delivery dict ...]
\end{verbatim}
\endgroup

which is probably easy enough.

\subsubsection{Simple Delivery}

Finally, there is:

\begin{quote}
\textbf{-delivery first}
\end{quote}

The return value is the value of the first formal parameter, or an empty string if there are no formal parameters; values of additional formal parameters are discarded. \textit{This is the default.}

This may seem a strange mode of delivery, and an even stranger thing to have as default, but it is what one must have if the \cite{tip372} examples of varying the \textit{argspec} should work as claimed there. With the same demo as before, one gets:
\begingroup\small\begin{verbatim}
 % coroutine C demo first
 % C "a b" ; # -arguments foo
 a b
 % C "a b" ; # -arguments {{arg ""}}
 a b
 % C       ; # -arguments {{arg ""}}
 % C "a b" ; # -arguments args
 {a b}
 % C "a b" ; # -arguments {foo args}
 a b
\end{verbatim}
\endgroup

An alternative which is perhaps more convenient, but also more complicated, is to have the default \textbf{-delivery} depend on the number of formal parameters. It is then likely that one for the case of more than one formal parameter would want Yet Another delivery mode...

\section{Reference Implementation}

None yet.

\section{Other Commands}

If coroutines are going to have \textbf{proc}-style argspecs, then it will probably make sense for \textbf{info args} and \textbf{info default} to operate on coroutine-commands as well as procedures.

\section{Acknowledgements}

This TIP is dedicated to Alexandre Ferrieux, who's known to have written that I should prioritize TIPping things over participating in tcl-core discussions. ;-)

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\begin{thebibliography}{TIP \#1}
\addcontentsline{toc}{section}{References}
\bibitem[TIP \#372]{tip372}
Colin McCormack,
\emph{Multi-argument Yield for Coroutines},
on-line at \url{http://purl.org/tcl/tip/372.html}
\end{thebibliography}
\end{document}
