\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#181: Add a [namespace unknown] Command}
\date{March 23, 2004}
\author{Neil Madden}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#181}&\textbf{Add a [namespace unknown] Command}\\\hline
Author&

Neil Madden \tipmail{nem}{cs.nott.ac.uk}
\\
Created&Tuesday, \(\text{23}^{\text{rd}}\) March 2004\\
Type&Project\\
Tcl Version&\(8.5\)\\
State&Final\\
Vote&Done\\
Version&\tipversion$Revision: 1.33 $\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposing adding a new \textbf{namespace} subcommand, \textbf{unknown}, which would register a per-namespace procedure for dealing with unknown commands.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Rationale}

There is an occassional need within Tcl scripts to change the way in which command names are resolved, for instance when implementing language constructs such as object systems or lexically-scoped commands. Many of these issues are addressed by the newly-accepted \textbf{namespace path} command \cite{tip229}. However, there are still a few situations where more customized behaviour is required. For instance, implementing custom per-namespace command auto-loading, or to use auto-expansion of leading word as in TOOT [\url{http://wiki.tcl.tk/11543}]. Currently, the only way to perform such customized command resolution behaviour is to override the global \textbf{::unknown} proc to install your custom handler. There are several drawbacks with this mechanism.

Firstly, it is difficult to override the global \textbf{::unknown} proc if you are writing a package, as good style dictates that you shouldn't override commands outside your package namespace without being explicitly asked to do so.

Secondly, as Tcl searches for a hard-coded fallback procedure name (\textbf{::unknown}), in order to override it's functionality you have to rename it and then install your own replacement --- and the new version becomes the default fallback behaviour for the entire application. In the case of implementing custom auto-loading behaviour, you may only want to override the behaviour for your namespace, and not for the entire interpreter. Currently, the only way to do this is to define a new \textbf{::unknown} procedure which does pattern matching on the command name it is passed.

Finally, if a package does override the \textbf{::unknown} procedure it has to be careful to save the old handler, and then invoke it for commands which it is not interested in. This is an error-prone approach, and results in a cascade of procedure calls, often with each one only interested in a subset of the commands being searched for.

\section{Related TIPs}

There have been two previous attempts at modifying Tcl's command resolution process. \cite{tip52} proposed that the search order be changed to traverse the complete namespace hierachy from most specific namespace to the most general (the global namespace). This TIP was withdrawn as it was not backwards compatible. \cite{tip142} proposed a global variable which would hold a namespace search path. This TIP was also withdrawn as it does not allow different namespaces to have different search paths. \cite{tip229} proposed the \textbf{namespace path} command as a way of specifying a per-namespace search path for command resolution. This TIP was accepted, and provides a good general mechanism for custom command resolution. The current TIP is complementary to \cite{tip229} and can be used to handle cases where more flexible behaviour is required.

\section{Proposed Change}

This TIP proposes that the handling of unknown commands be done on a per-namespace basis through the introduction of an \textbf{unknown} subcommand of the \textbf{namespace} command.

\begin{quote}
\textbf{namespace unknown} ?\textit{commandPrefix}?
\end{quote}

The subcommand would accept either zero or one argument(s), and is similar in interface to the \textbf{interp bgerror} command added by \cite{tip221}. If no arguments are given, the command returns the handler for the current namespace. The optional argument \textit{commandPrefix} is a command (strictly a prefix list consisting of a command and optional arguments) to execute if normal command lookup from the current namespace fails. The command will be concatenated with the full invocation line of the command being searched for (i.e. the command name and all arguments), and evaluated in the scope of the current namespace. The first word in the list given must be a command name which must be able to be resolved without resorting to the unknown mechanism (i.e. it must either be a command in the current or global namespace, or be fully-qualified). If this cannot be done, a stock error message will be generated referring to the original unknown command (and \textit{not} the missing handler) --- this is how Tcl currently behaves if no \textbf{::unknown} procedure exists.

The command resolution procedure would be altered from this:
\begin{enumerate}
\item{}
Lookup command in current namespace.
\setcounter{enumi}{1}
\item{}
If that fails, use path supplied by \textbf{namespace path}.
\setcounter{enumi}{2}
\item{}
If that fails, lookup command in global namespace.
\setcounter{enumi}{3}
\item{}
If that fails, call global \textbf{::unknown} procedure.
\end{enumerate}

to this:
\begin{enumerate}
\item{}
Lookup command in current namespace.
\setcounter{enumi}{1}
\item{}
If that fails, use path supplied by \textbf{namespace path}.
\setcounter{enumi}{2}
\item{}
If that fails, lookup command in global namespace.
\setcounter{enumi}{3}
\item{}
If that fails, call the unknown handler for the namespace in which the unknown command was invoked.
\end{enumerate}

Note that this TIP does not change (or allow changing) the default command resolution procedure --- the current, namespace path, and global namespaces are always searched before the unknown handler is called. This is so that resolution of the unknown handler itself can be performed, and so that the handler can be implemented without resorting to fully qualifying every command in it (e.g. having to use ::set).

It also should be noted that the unknown handler that is called is for the namespace that \textit{invoked} the unknown command, rather than the namespace which is the \textit{target} of the call. There are a number of reasons for this decision: Firstly, there is no guarantee that the target namespace actually exists (this might be a reason why the command was not found). In that case, there would be no target namespace unknown handler, and we would be forced to fallback on the global default. Secondly, the mechanism is designed mainly for namespace authors who wish to implement some custom behaviour that affects the operation of their own code, e.g., custom auto-loading of missing commands, or more sophisticated command lookup procedures, etc. This suggests that the responsibility for dealing with unknown commands should fall on the originating namespace, rather than being placed on arbitrary other namespaces. It is believed that this is the most sensible, and the most predictable, behaviour for an unknown command mechanism. In addition, with the introduction of \textbf{namespace ensemble} \cite{tip112} there now exists a flexible mechanism for handling the opposite behaviour, allowing the target namespace to handle requests for unknown commands (e.g., forwarding requests). The author of the present TIP considers this to be the right division of responsibility, as ensembles should become the default mechanism for accessing public operations of external namespaces, and also an ensemble command can be guaranteed that the target namespace does indeed exist.

The default unknown handler for the global namespace is a handler called \textbf{::unknown}. The default handler for other namespaces calls the global unknown handler. This means that by default, we have exactly the same mechanism that exists currently in Tcl. In order to change the mechanism for an individual namespace, you may register a new unknown handler for that namespace. When no handler is registered for a namespace, then a call to \textbf{namespace unknown} will return an empty string (for non-global namespaces) or \textbf{::unknown} for the global namespace. This is so that a distinction can be made between namespaces which have no handler set, and namespaces which have had an unknown handler called \textbf{::unknown} deliberately registered for them. With this scheme it is possible to set a global per-interpreter unknown command handler by setting the unknown handler for the global namespace. This can then be overridden on a per-namespace basis, if required.

The calling of unknown handlers registered with \textbf{namespace unknown} would be identical to the current calling of the \textbf{::unknown} procedure --- the handler will be called with the command name and all of its arguments, as it was originally invoked. The handler should be a valid Tcl list representing a command and possible initial arguments. For instance, a single handler proc could be used for several namespaces with the namespace passed as an argument:
\begingroup\small\begin{verbatim}
 proc handleunknown {ns cmd args} { ... }
 namespace eval foo { namespace unknown [list ::handleunknown ::foo] }
 namespace eval bar { namespace unknown [list ::handleunknown ::bar] }
\end{verbatim}
\endgroup

Setting the unknown handler to \{\} (an empty list) restores the default handler (\textbf{::unknown} for global namespace, global unknown handler for all other namespaces).

\subsection{C API}

Additionally, this TIP proposes adding two new public functions to the Tcl C-API to expose this functionality at the C-level. The proposed new functions are:
\begingroup\small\begin{verbatim}
 Tcl_Obj *Tcl_GetNamespaceUnknownHandler(Tcl_Interp *interp,
                                         Tcl_Namespace *nsPtr);
\end{verbatim}
\endgroup

Returns the current unknown command handler registered for the given namespace, or NULL if none is.
\begingroup\small\begin{verbatim}
 int Tcl_SetNamespaceUnknownHandler(Tcl_Interp *interp,
                                    Tcl_Namespace *nsPtr,
                                    Tcl_Obj *handlerPtr);
\end{verbatim}
\endgroup

Sets the unknown command handler for the namespace, or resets it to the default if \textit{handlerPtr} is NULL or an empty list.

\subsection{Consequences}

As a final note, there is a useful side-effect to always resolving the unknown handler itself in the current namespace, in that an unknown handler can be registered for the global namespace which is not fully qualified, and it will be resolved relative to the namespace in which an unknown command is invoked. To illustrate:
\begingroup\small\begin{verbatim}
 # Set global unknown handler to unqualified name
 namespace unknown unknown
 namespace eval foo { proc unknown {args} { puts "FOO" } }
 proc unknown {args} { puts "GLOBAL" }
 
 bar ;# prints GLOBAL
 namespace eval foo { bar } ;# prints FOO
 namespace eval other { bar } ;# prints GLOBAL
\end{verbatim}
\endgroup

\section{Reference Implementation}

A reference implementation is available attached to Patch 958222 on the Tcl project at SourceForge [\url{http://sf.net/tracker/?func=detail&aid=958222&group_id=10894&atid=310894}].

The current patch is called tip181-4.patch

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\begin{thebibliography}{TIP \#5}
\addcontentsline{toc}{section}{References}
\bibitem[TIP \#52]{tip52}
David Cuthbert,
Andreas Kupries,
\emph{Hierarchical Namespace Lookup of Commands and Variables},
on-line at \url{http://purl.org/tcl/tip/52.html}
\bibitem[TIP \#112]{tip112}
Donal K. Fellows,
\emph{Ensembles are Namespaces are Commands},
on-line at \url{http://purl.org/tcl/tip/112.html}
\bibitem[TIP \#142]{tip142}
Ulrich Schoebel,
\emph{Search Path Variable to Lookup Command Names in Namespaces},
on-line at \url{http://purl.org/tcl/tip/142.html}
\bibitem[TIP \#221]{tip221}
Don Porter,
\emph{Allow Background Error Handlers to Accept Return Options},
on-line at \url{http://purl.org/tcl/tip/221.html}
\bibitem[TIP \#229]{tip229}
Donal K. Fellows,
\emph{Scripted Control of Name Resolution in Namespaces},
on-line at \url{http://purl.org/tcl/tip/229.html}
\end{thebibliography}
\end{document}
