\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#297: Integer Type Introspection and Conversion}
\date{November 20, 2006}
\author{Don Porter}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#297}&\textbf{Integer Type Introspection and Conversion}\\\hline
Author&

Don Porter \tipmail{dgp}{users.sf.net}
\\
Created&Monday, \(\text{20}^{\text{th}}\) November 2006\\
Type&Project\\
Tcl Version&\(8.7\)\\
State&Draft\\
Vote&Pending\\
Version&\tipversion$Revision: 1.3 $\\
Keywords&Tcl, number, expression\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposes changes to complete the set of commands to test and convert among Tcl's integer types.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Background}

There are four integer types that appear in Tcl's C API. They are \textit{int}, \textit{long}, \textit{Tcl\_WideInt}, and \textit{mp\_int}. The corresponding routines to pull a value of each of those types from a \textit{Tcl\_Obj} are \textbf{Tcl\_GetIntFromObj}, \textbf{Tcl\_GetLongFromObj}, \textbf{Tcl\_GetWideIntFromObj}, and \textbf{Tcl\_GetBignumFromObj}. These integer types form increasing sets. That is, every \textit{Tcl\_Obj} that can return an \textit{int} can also return a \textit{long}, \textit{Tcl\_WideInt}, or \textit{mp\_int}.

Strictly speaking, the set of \textit{Tcl\_Obj} values that can successfully return either and \textit{int}, \textit{long}, or \textit{Tcl\_WideInt} is platform-dependent, because the size of these types is platform dependent. \textbf{Tcl\_GetIntFromObj} accepts integer values in any format (decimal, binary, octal, hexadecimal, etc., see \textbf{TCL\_PARSE\_INTEGER\_ONLY} in \cite{tip249}) that are within the inclusive platform-dependent range (\textbf{-UINT\_MAX}, \textbf{UINT\_MAX}). \textbf{Tcl\_GetLongFromObj} accepts integer values in any format that are within the inclusive platform-dependent range (\textbf{-ULONG\_MAX}, \textbf{ULONG\_MAX}). \textbf{Tcl\_GetWideIntFromObj} accepts integer values in any format that are within the inclusive platform-dependent range (\textbf{-ULLONG\_MAX}, \textbf{ULLONG\_MAX}), or the appropriate equivalent for the platform. \textbf{Tcl\_GetBignumFromObj} accepts integer values in any format with (effectively) no limit on range.

The most common example of platform dependence of results seen at the script level is the different results of \textbf{[expr int(.)]} on most 32-bit systems,
\begingroup\small\begin{verbatim}
 % set tcl_platform(wordSize)
 4
 % expr int(1<<31)
 -2147483648
\end{verbatim}
\endgroup

compared with LP64 systems.
\begingroup\small\begin{verbatim}
 % set tcl_platform(wordSize)
 8
 % expr int(1<<31)
 2147483648
\end{verbatim}
\endgroup

These differences show up most unfortunately when implementing algorithms designed to operate explicitly on 32-bit buffers, where the only portable way to do the operations in Tcl is with careful application of masking (\textit{\& 0xffffffff}). For one well-known example, see the \textit{sha1} package in tcllib. The additional operations in Tcl expressions harm performance.

There are other Tcl routines that pull values from \textit{Tcl\_Obj} that accept supersets of one of the integer types. An example is \textbf{Tcl\_GetIndexFromObj} which will accept anything that \textbf{Tcl\_GetIntFromObj} accepts, as well as other string values. There are also Tcl built-in commands that accept arguments that are supersets of one of the integer types. An example is \textbf{uplevel} which accepts as its level argument anything that \textbf{Tcl\_GetIntFromObj} accepts, as well as other string values.

All Tcl commands are ultimately defined by the C command procedures that run to implement them, and when those command procedures use the routines mentioned above to pull values from command arguments, the result is that the Tcl commands will succeed or fail depending on whether or not an integer value of the right type has been provided by the caller. As a simple example:
\begingroup\small\begin{verbatim}
 % lindex {} 0xffffffff
 % lindex {} 0x100000000
 bad index "0x100000000": must be integer or end?-integer?
\end{verbatim}
\endgroup

In order to avoid errors from commands, a cautious programmer may wish to test whether a value is of an acceptable type before passing it to a command. The \textbf{string is integer} command has long offered this facility for commands that require (a superset of) an \textit{int}.
\begingroup\small\begin{verbatim}
 % string is integer 0xffffffff
 1
 % string is integer 0x100000000
 0
\end{verbatim}
\endgroup

Most of Tcl's built-in commands that accept an integer valued argument require that argument to be acceptable to \textbf{Tcl\_GetIntFromObj} and the existing \textbf{string is integer} command provides sufficient introspection.

\cite{tip188} created the new command \textbf{string is wideinteger}, and that is suitable for testing values for the small number of Tcl commands that strictly require a value acceptable to \textbf{Tcl\_GetWideIntFromObj}. Those commands are:
\begingroup\small\begin{verbatim}
 after $wide
 binary format w $wide
 chan seek $chan $wide
 chan truncate $chan $wide
 clock add $wide
 clock format $wide
\end{verbatim}
\endgroup

There are some built-in Tcl commands that require an argument that is acceptable to \textbf{Tcl\_GetBignumFromObj}. That is, the argument must be an integer, but no range limit is imposed. Currently there is no test command appropriate for argument checking for these commands.
\begingroup\small\begin{verbatim}
 dict incr $dictVar $bignumkey $bignum
 expr srand($bignum)
 expr ~$bignum
 expr $bignum % $bignum
 expr $bignum << $int
 expr $bignum >> $bignum
 expr $bignum & $bignum
 expr $bignum ^ $bignum
 expr $bignum | $bignum
 incr $bignumVar $bignum
 format $integerSpecifier $bignum
\end{verbatim}
\endgroup

There are some built-in Tcl commands that require an argument that is acceptable to \textbf{Tcl\_GetLongFromObj}. Currently there is no test command appropriate for argument checking for these commands.
\begingroup\small\begin{verbatim}
 binary format i $long
 binary format s $long
 binary format c $long
 file atime $path $long
 file attributes $path -permissions $long
 file mtime $path $long
\end{verbatim}
\endgroup

Note that the accepted ranges of the \textbf{Tcl\_GetFooFromObj} routines can lead to surprising results. For example, \textbf{Tcl\_GetIntFromObj} accepts values from \textbf{-UINT\_MAX} to \textbf{UINT\_MAX}. For some things this is good, since it supports
\begingroup\small\begin{verbatim}
 binary format i 0x80000000
\end{verbatim}
\endgroup

on 32-bit platforms, which is a common coding style. However the same range acceptance leads to surprising (and arguably incorrect, in the presence of bignum support) things like:
\begingroup\small\begin{verbatim}
 % string repeat a -4294967290
 aaaaaa
\end{verbatim}
\endgroup

It seems there are good uses for both strict and liberal routines for pulling integer ranges from a \textit{Tcl\_Obj}. Compatibility concerns would favor keeping the existing routines liberal, and adding strict counterparts. If this is pursued, however, another collection of \textbf{string is} test commands would be needed as well.

\section{Proposed Changes}

Still pondering how best to react to this background. Discussion invited on TCLCORE.

\section{Compatibility}

\section{Reference Implementation}

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\begin{thebibliography}{TIP \#2}
\addcontentsline{toc}{section}{References}
\bibitem[TIP \#188]{tip188}
Kevin Kenny,
\emph{Add 'string is wideinteger' to the 'string is' Subcommand},
on-line at \url{http://purl.org/tcl/tip/188.html}
\bibitem[TIP \#249]{tip249}
Kevin B. Kenny,
David S. Cargo,
Don Porter,
\emph{Unification of Tcl's Parsing of Numbers},
on-line at \url{http://purl.org/tcl/tip/249.html}
\end{thebibliography}
\end{document}
