\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#357: Export TclLoadFile}
\date{October 01, 2009}
\author{Kevin Kenny}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#357}&\textbf{Export TclLoadFile}\\\hline
Author&

Kevin Kenny \tipmail{kevin.b.kenny}{gmail.com}
\\
Created&Thursday, \(\text{1}^{\text{st}}\) October 2009\\
Type&Project\\
Tcl Version&\(8.6\)\\
State&Final\\
Vote&Done\\
Version&\tipversion$Revision: 1.14 $\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposes to promote the internal calls, \textit{TclLoadFile} and \textit{TclpFindSymbol} to the external API, making them available to C extensions. In addition, it proposes to introduce a \textit{Tcl\_FSUnloadFile} to the VFS layer, removing the weird duplicate client data from \textit{TclLoadFile} and wrapping all necessary data into the \textbf{Tcl\_LoadHandle}
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Rationale}

In developing TDBC \cite{tip308}, the author of this TIP was advised to look at the way that the 'oratcl' extension contrives to build on a system where Oracle is not installed as a model for TDBC extensions. Examination of the code revealed that it operates essentially by constructing at run time a Stubs table for the routines in the Oracle client library. There is a maze of \textbf{\#if} directives selecting whether this process is accomplished by the system calls that manage Unix .so files (\textit{dlopen} and friends), Windows .DLL files (\textit{LoadLibrary} and related calls), HP-UX .shl files (\textit{shl\_load,} etc.), and so on.

Tcl already has functionality so that a caller can abstract away all this complexity. It provides the capability in the \textit{TclLoadFile} call, but this call is \textbf{MODULE\_SCOPE} and not exported even in the internal Stubs table. For this reason, it is entirely unavailable to TEA-compliant extensions.

If this call were available, it would be feasible, in the 8.6 time frame, to bundle all the database-specific TDBC drivers with the core TDBC distribution, since things could be set up so that they will build anywhere, even in the absence of the databases where they connect.

However, this call was never fully rationalized in the VFS world. Its strange API (with both a \textbf{Tcl\_LoadHandle} and a \textbf{ClientData}, and a function pointer for unloading the library) is, as several reviewers pointed out, not somehting that we want to make available to general callers. Hence, a few more routines need rework.

\section{Specification}

The \textit{TclLoadFile} call shall be renamed \textit{Tcl\_LoadFile} and exported in the external Stubs. Its call signature shall be changed to:

\begin{quote}
EXTERN int \textbf{Tcl\_LoadFile}( Tcl\_Interp *\textit{interp}, Tcl\_Obj *\textit{pathPtr}, const char *\textit{symbols}[], int \textit{flags}, void *\textit{procPtrs}, Tcl\_LoadHandle *\textit{handlePtr});
\end{quote}

In this call, \textit{interp} designates an interpreter for error reporting. \textit{pathPtr} is an object containing the path of the library to be loaded. If \textit{pathPtr} is a single name, the library search path of the current environment will be used to resolve it. The \textit{symbols} array contains a NULL-terminated set of character strings that are the names of the imported symbols. \textit{symbols} may be NULL if the library is to be loaded without resolving any symbols.

The return value of \textit{Tcl\_LoadFile} is a standard Tcl result code. If the result is TCL\_ERROR, the interpreter result will contain an appropriate error message.

On return, \textit{Tcl\_LoadFile} fills in the block of memory commencing at the address given by \textit{procPtrs} with the addresses that correspond to the names in \textit{symbols.} If a name cannot be resolved in the given library, an error will be returned and all entries in \textit{procPtrs} will be invalid. \textit{procPtrs} may be NULL if no symbols are to be resolved.

On a successful return, the \textit{loadHandle} pointer will be a handle suitable for passing to \textit{Tcl\_FindSymbol} for resolving additional symbols in the library, or to \textit{Tcl\_FSUnloadFile} to unload the library.

The \textit{TclpFindSymbol} call shall be renamed \textit{Tcl\_FindSymbol} and exported in the external Stubs. Its call signature shall be:

\begin{quote}
EXTERN void* \textbf{Tcl\_FindSymbol}( Tcl\_Interp *\textit{interp}, Tcl\_LoadHandle *\textit{loadHandle}, const char *\textit{symbol});
\end{quote}

This call searches for the given \textit{symbol} in the already-loaded library identified by \textit{loadHandle}. If the symbol is found, a pointer to its memory address is returned. Otherwise, NULL is returned, and an error message is left in \textit{interp} (if \textit{interp} is not NULL).

A new call, \textit{Tcl\_FSUnloadFile} shall be introduced and exported in the external Stubs.Its call signature shall be:

\begin{quote}
EXTERN int \textbf{Tcl\_FSUnloadFile}( Tcl\_Interp *\textit{interp}, Tcl\_LoadHandle *\textit{loadHandle});
\end{quote}

This call unloads the library identified by \textit{loadHandle}. It differs from the [unload] command in that no 'unload' procedure is called; the library is simply unloaded without first being given a chance to clean itself up. (This function is a lower-level interface used by [unload] as part of doing its work.) The return value is either TCL\_OK or TCL\_ERROR; when TCL\_ERROR is returned, an appropriate message is left in the result of \textit{interp} (if \textit{interp} is not NULL).

\section{Internals}

The \textbf{Tcl\_LoadHandle} object shall be represented internally by a structure, declared in \tiplangle{}tclInt.h\tiprangle{}, looking like:
\begingroup\small\begin{verbatim}
 struct Tcl_LoadHandle_ {
     ClientData clientData;	/* Client data is the load handle in the
 				 * native filesystem if a module was loaded
 				 * there, or an opaque pointer to a structure
 				 * for further bookkeeping on load-from-VFS
 				 * and load-from-memory */
     TclFindSymbolProc* findSymbolProcPtr;
 				/* Procedure that resolves symbols in a
 				 * loaded module */
     Tcl_FSUnloadFileProc* unloadFileProcPtr;
 				/* Procedure that unloads a loaded module */
 }
\end{verbatim}
\endgroup

The \textit{Tcl\_FindSymbolProc} and \textit{Tcl\_FSUnloadFileProc} data types are declared to be functions of the same type signature as \textit{Tcl\_FindSymbol} and \textit{Tcl\_FSUnloadFile}.

Virtual file systems that implement the \textit{loadFileProc} are responsible for ensuring that their \textit{'Tcl\_LoadHandle} that the \textit{loadFileProc} returns conforms with this convention. As far as the author has been able to determine, no non-Core filesystem provides anything but NULL for the \textit{loadFileProc}. Certainly, tclvfs and trofs do not. Most other virtual filesystems layer atop tclvfs. 

\section{Reference Implementation}

A reference implementation is nearing completion, and the draft code is available attached to [Patch 2891616][\url{https://sourceforge.net/support/tracker.php?aid=2891616}] at SourceForge.

\section{License}

This file is explicitly released to the public domain and the author explicitly disclaims all rights under copyright law.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\begin{thebibliography}{TIP \#1}
\addcontentsline{toc}{section}{References}
\bibitem[TIP \#308]{tip308}
Kevin B. Kenny,
Artur Trzewik,
Andreas Leitgeb,
Donal K. Fellows,
\emph{Tcl Database Connectivity (TDBC)},
on-line at \url{http://purl.org/tcl/tip/308.html}
\end{thebibliography}
\end{document}
