\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#314: Ensembles with Parameters}
\date{February 26, 2008}
\author{Lars Hellström}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#314}&\textbf{Ensembles with Parameters}\\\hline
Author&

Lars Hellström \tipmail{Lars.Hellstrom}{residenset.net}
\\
Created&Tuesday, \(\text{26}^{\text{th}}\) February 2008\\
Type&Project\\
Tcl Version&\(8.6\)\\
State&Final\\
Vote&Done\\
Version&\tipversion$Revision: 1.5 $\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposes that \textbf{namespace ensemble} commands are generalised so that they may have arguments before the subcommand name.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Rationale}

The introduction of \textbf{\{*\}} for argument expansion has made it much more convenient to use command prefixes for callbacks. One particular idiom that command prefixes provide for is ``ClientData'' arguments, i.e., the same command is used for several different callbacks, as exactly what it does or acts on is controlled by the extra argument(s). Of course, both command prefixes and the ``ClientData'' idiom are already the rule for callbacks from the core, but \textbf{\{*\}} will most likely make them more common also for callbacks from Tcl code.

A disadvantage of this idiom is however that it currently cannot be used if the base command of a command prefix is to be an ensemble, as the subcommand name in an ensemble must follow immediately after the base command name. Callbacks from the core which take a subcommand are rare --- the only obvious example is the reflected channel callback command --- but in higher level code such callbacks are fairly common. Using namespace ensembles for implementing such callbacks makes the code much more modular than using a procedure would. Hence it is desirable to remove the restriction that the subcommand name of a namespace ensemble must appear as the first argument, and instead allow there to be some number of ``parameter'' arguments between the base command name and the subcommand name.

One application of parameter arguments is to use namespace ensembles as a white-box OO system, where the parameters hold the state of the object. More explicitly, each object instance in such a system would be a command prefix consisting of (i) one ensemble command that handles the method dispatch and (ii) the necessary number of parameter arguments, whose values make up the state of the object. Being values, such objects are necessarily immutable; it is however possible to define methods which return a mutated form of the object. For light-weight objects, this system has the advantage that instances do not have to be destroyed explicitly, but of course that also means that they cannot own any resources that require explicit destruction.

Some may argue that ensemble parameters are not necessary because any client data can be embedded into the prefixes of the \textbf{-map} dictionary of the ensemble. This is however only true to the extent that multiword command prefixes themselves are unnecessary; it is similarly possible to embed extra arguments into an \textbf{interp alias}. Both of these ``solutions'' have the disadvantage that they create an auxiliary command which one must explicitly dispose of or leak memory, whereas all memory used by a command prefix is automatically released when the last reference to it goes away.

\section{Specification}

A new ensemble option \textbf{-parameters} is introduced, which takes a list of parameter names as value and defaults to the empty list. Two C functions for setting and getting the value of this option are added to the public stubs table:

\begin{quote}
int \textbf{Tcl\_SetEnsembleParameterList}(Tcl\_Interp *\textit{interp}, Tcl\_Command \textit{token}, Tcl\_Obj *\textit{paramList})
\end{quote}

\begin{quote}
int \textbf{Tcl\_GetEnsembleParameterList}(Tcl\_Interp *\textit{interp}, Tcl\_Command \textit{token}, Tcl\_Obj **\textit{paramListPtr})
\end{quote}

(This is the same pattern as for the other ensemble options, e.g. for the \textbf{-subcommands} option's implementation.)

The general structure of a namespace ensemble command call will have the form:

\begin{quote}
\textit{baseCmd} \{*\}\textit{parameterArgs} \textit{subCmd} \{*\}\textit{otherArgs}
\end{quote}

where the number of arguments between the base command and the subcommand is exactly the same as the number of elements in the value of the \textbf{-parameters} option. It is an error to call the \textit{baseCmd} with fewer arguments than the number of parameters plus one. If \textit{cmdPrefix} is the command prefix to which the ensemble \textit{baseCmd} maps the \textit{subCmd}, then the above call gets translated into

\begin{quote}
\{*\}\textit{cmdPrefix} \{*\}\textit{parameterArgs} \{*\}\textit{otherArgs}
\end{quote}

\section{Examples}

An ensemble for arithmetic in integer-modulo-\textit{n} rings can be implemented as follows:
\begingroup\small\begin{verbatim}
 namespace eval intmod {
     proc + {n args} {expr {[::tcl::mathop::+ {*}$args] % $n}}
     proc - {n args} {expr {[::tcl::mathop::- {*}$args] % $n}}
     proc * {n args} {expr {[::tcl::mathop::* {*}$args] % $n}}
     proc / {n a b} {
         set c $n
         set r 0
         set s 1
         while {$b} {
             set q [expr {$c / $b}]
             set b [expr {$c - $q*[set c $b]}]
             set s [expr {$r - $q*[set r $s]}]
         }
         if {$a % $c == 0} then {
             return [expr {$r * $a / $c % $n}]
         } else {
             return -code error "No such quotient"
         }
     }
     proc 0 {n} {return 0}
     proc 1 {n} {return 1}
     namespace ensemble create -parameters n -subcommands {+ - * / 0 1}
     # That [namespace export] takes patterns as arguments starts
     # feeling somewhat corny when * is a common command names.
 }
\end{verbatim}
\endgroup

Some example results:
\begingroup\small\begin{verbatim}
 % intmod 7 + 4 4
 1
 % intmod 7 - 1
 6
 % intmod 7 * 3 5
 1
 % intmod 7 / 3 2
 5
 % intmod 32003 / 3 2
 16003
 % intmod 32768 / 3 2
 No such quotient
 % intmod
 wrong # args: should be "intmod n subcommand ?argument ...?"
\end{verbatim}
\endgroup

An ensemble for matrix arithmetic over some ring can be implemented as follows:
\begingroup\small\begin{verbatim}
 namespace eval matrix {
     proc + {ring A B} {
         if {[llength $A] != [llength $B] ||
             [llength [lindex $A 0]] != [llength [lindex $B 0]]} then {
             return -code error -errorcode {ARITH DOMAIN}
               "Matrix shapes do not match"
         }
         set res {}
         foreach a_row $A b_row $B {
             set r_row {}
             foreach a $a_row b $b_row {
                 lappend r_row [{*}$ring + $a $b]
             }
             lappend res $r_row
         }
         return $res
     }
     proc - {ring A B} {
         if {[llength $A] != [llength $B] ||
             [llength [lindex $A 0]] != [llength [lindex $B 0]]} then {
             return -code error -errorcode {ARITH DOMAIN}
               "Matrix shapes do not match"
         }
         set res {}
         foreach a_row $A b_row $B {
             set r_row {}
             foreach a $a_row b $b_row {
                 lappend r_row [{*}$ring - $a $b]
             }
             lappend res $r_row
         }
         return $res
     }
     proc * {ring A B} {
         if {[llength [lindex $A 0]] != [llength $B]} then {
             return -code error -errorcode {ARITH DOMAIN}
               "Matrix shapes do not match"
         }
         set res {}
         foreach a_row $A {
             set r_row {}
             foreach a $a_row b_row $B {
                 set r [{*}$ring 0]
                 foreach b $b_row {
                     set r [{*}$ring + $r [{*}$ring * $a $b]]
                 }
                 lappend r_row $r
             }
             lappend res $r_row
         }
         return $res
     }
     # ...
     namespace export *
     namespace ensemble create -parameters ringCmdPrefix
 }
\end{verbatim}
\endgroup

Some more example results:
\begingroup\small\begin{verbatim}
 % set A {{1 2} {3 4}}
 % matrix {intmod 7} + $A $A
 {2 4} {6 1}
 % set B {{0 2} {1 3}}
 % matrix {intmod 100} * $A $B
 {2 8} {6 16}
 % matrix {intmod 5} * $A $B
 {2 3} {1 1}
 % matrix {intmod 5} - $A $B
 {1 0} {2 1}
 % matrix
 wrong # args: should be "matrix ringCmdPrefix subcommand ?argument ...?"
\end{verbatim}
\endgroup

In the same way, one can define a \textbf{polynomial} ensemble for arithmetic with polynomials over some ring. Then one can immediately start doing calculations with e.g. matrices whose coefficients are polynomials over integers modulo 2, simply by using the command prefix
\begingroup\small\begin{verbatim}
 matrix {polynomial {intmod 2}}
\end{verbatim}
\endgroup

Composing constructions this way is a surprisingly quick way of implementing rather complex mathematical structures!

A trivial mutable object class can be implemented as follows:
\begingroup\small\begin{verbatim}
 namespace eval mutable_ns {
     proc get {value} {return $value}
     proc set {value newvalue} {list [namespace current] $newvalue}
     namespace export get set
     namespace ensemble create -parameters value
 }
 proc mutable {initval} {
     list [namespace which -command mutable_ns] $initval
 }
\end{verbatim}
\endgroup

Some example results:
\begingroup\small\begin{verbatim}
 % set a [mutable 0]
 ::mutable_ns 0
 % {*}$a get
 0
 % {*}$a foo
 unknown or ambiguous subcommand "foo": must be get, or set
 % set b [{*}$a set 3] ; # Creates a modified copy
 ::mutable_ns 3
 % {*}$b get
 3
 % {*}$a get
 0
\end{verbatim}
\endgroup

\section{Rejected Alternatives}

Most of the time, only the number of parameters is relevant; their names are merely used when throwing a ``wrong \# args'' error. Hence an alternative would be to have taken that number as the value of the \textbf{-parameters} option, but requesting a list of names encourages the programmer to provide more information available for introspection and should help to produce better error messages.

An alternative principle for forming the mapped-to command could be that the parameters should remain in the same position in the command. This would mean that rather than mapping

\begin{quote}
\textit{baseCmd} \{*\}\textit{parameterArgs} \textit{subCmd} \{*\}\textit{otherArgs}
\end{quote}

to

\begin{quote}
\{*\}\textit{cmdPrefix} \{*\}\textit{parameterArgs} \{*\}\textit{otherArgs}
\end{quote}

one would map it to

\begin{quote}
[\textbf{lindex} \textit{cmdPrefix} 0] \{*\}\textit{parameterArgs} \{*\}[\textbf{lrange} \textit{cmdPrefix} 1 end] \{*\}\textit{otherArgs}
\end{quote}

but this is slightly more complicated to do, and it seems less useful. For example, this would prevent using an \textbf{apply} \textit{lambda} form of \textit{cmdPrefix} in an ensemble with parameters.

\section{Future Extensions}

In analogy with the \textbf{-unknown} handler for an ensemble, it might be useful to have a handler for ensembles being called with \textit{too few} arguments; it is not uncommon for ensemble-like commands that one in certain cases can omit the subcommand name. Possibly this functionality could even be integrated into the \textbf{-unknown} handler. There is however nothing in that which is directly related to the issue of ensembles having parameters, other than that parameters make it possible to call an ensemble command with way too few arguments instead of just one too few.

\section{Reference Implementation}

A reference implementation is available as SF Tcl patch \#1901783. [\url{https://sourceforge.net/support/tracker.php?aid=1901783}]

\subsection{Notes}

One detail in this implementation which might require further consideration because it results in script-level-visible behaviour is the matter of how the list of parameter names is turned into error messages. Currently that is done by (effectively) \textbf{join}ing the list elements, but a possible alternative is to use the string representation of the list. Joining seems to give better control to the user of what gets put in the message, but the results are probably equivalent for all alphanumeric choices of parameter names.

Deep down, this touches upon the matter of how the user may distinguish actual argument values from formal argument names in syntax error messages. As far as I can tell, there currently isn't a way of doing that, but perhaps there should be. In want of clear rules for this, the reference implementation doesn't seem to fare any worse than what is already in the core.

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\end{document}
