\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#115: Making Tcl Truly 64-Bit Ready}
\date{October 23, 2002}
\author{Donal K. Fellows}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#115}&\textbf{Making Tcl Truly 64-Bit Ready}\\\hline
Author&

Donal K. Fellows \tipmail{donal.k.fellows}{man.ac.uk}
\\
Created&Wednesday, \(\text{23}^{\text{rd}}\) October 2002\\
Type&Project\\
Tcl Version&\(9.0\)\\
State&Draft\\
Vote&Pending\\
Version&\tipversion$Revision: 1.3 $\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposes changes to Tcl to make it operate more effectively on 64-bit systems.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Rationale}

It is a fact of life that 64-bit platforms are becoming more common. While once the assumption that virtually everything was a 32-bit machine (where not smaller) was valid, this is no longer the case. Particularly on modern supercomputers (though increasingly in workstations and high-end desktop systems too), the amount of memory that the machine contains is exceeding 2GB, and the need to address very large amounts of memory is certainly there in scientific and engineering applications. And where they lead, consumer systems will probably follow too.

At the moment, Tcl is ill-prepared for this. In particular, the type used for expressing sizes of entities in Tcl (whether strings, lists or undifferentiated blocks of memory) is \textit{int} (and cannot be made into an \textit{unsigned int} in most of those places where it is not already an unsigned value) but on the majority of 64-bit platforms this is still a 32-bit type, which is a major restriction. However, on the vast majority of those platforms \textit{long} is a 64-bit type, and so a suitable replacement. (The exceptions to this are the Alpha --- but that is unusual in that both \textit{int} and \textit{long} are 64-bit types there, meaning that the platform will be unaffected by such an alteration --- and Win64, which has a 32-bit \textit{long} but 64-bit pointers.)

Luckily, standards like POSIX have already been dealing with this problem before us, and the types \textit{size\_t} (which is unsigned) and \textit{ssize\_t} (which is signed) exist for the sorts of uses we're interested in (i.e. they are both the same size as each other, and \textit{size\_t} is large enough to describe the size of any allocatable memory chunk.)

\section{Details of Changes}

The key changes will be to change the lengths of the following types from \textit{int} to \textit{ssize\_t} in all appropriate places, and \textit{unsigned int} to \textit{size\_t} likewise (mainly in memory allocation routines.)
\begin{itemize}
\item{}
\textit{Tcl\_Obj} --- the \textit{length} member. (Potentially the \textit{refCount} member needs updating as well, but that's less critical.)
\item{}
\textit{Tcl\_SavedResult} --- the \textit{appendAvl} and \textit{appendUsed} members.
\item{}
\textit{Tcl\_DString} --- the \textit{length} and \textit{spaceAvl} members.
\item{}
\textit{Tcl\_Token} --- the \textit{size} and \textit{numComponents} members.
\item{}
\textit{Tcl\_Parse} --- the \textit{commentSize}, \textit{commandSize}, numWords\textit{, }numTokens\textit{ and }tokensAvailable\(''\) members.
\item{}
\textit{CompiledLocal} --- the \textit{nameLength} member.
\item{}
\textit{Interp} --- the \textit{appendAvl}, \textit{appendUsed} and \textit{termOffset} members.
\item{}
\textit{List} --- the \textit{maxElemCount} and \textit{elemCount} members.
\item{}
\textit{ByteArray} --- the \textit{used} and \textit{allocated} members.
\item{}
\textit{SortElement} --- the \textit{count} member.
\item{}
\textit{SortInfo} --- the \textit{index} member.
\item{}
\textit{CopyState} --- the \textit{toRead} and \textit{total} members.
\item{}
\textit{GetsState} --- the \textit{rawRead}, \textit{bytesWrote}, \textit{charsWrote} and \textit{totalChars} members.
\item{}
\textit{ParseInfo} --- the \textit{size} member.
\item{}
\textit{String} --- the \textit{numChars} member (see also the \textit{TestString} structure.)
\end{itemize}

Changes to the bytecode-related structures might be worthwhile doing too, though there are more backward-compatibility issues there.

These changes will force many of the types used in the public API to change as well. Notable highlights:
\begin{itemize}
\item{}
\textit{Tcl\_Alloc} will now take an \textit{size\_t}.
\item{}
\textit{Tcl\_GetByteArrayFromObj} will now take a pointer to a \textit{ssize\_t}.
\item{}
\textit{Tcl\_GetStringFromObj} will now take a pointer to a \textit{ssize\_t}.
\item{}
\textit{Tcl\_ListObjLength} will now take a pointer to a \textit{ssize\_t}.
\item{}
\textit{Tcl\_GetUnicodeFromObj} will now take a pointer to a \textit{ssize\_t}.
\end{itemize}

In the internal API, the following notable change will happen:
\begin{itemize}
\item{}
\textit{TclGetIntForIndex} will now take a pointer to a \textit{ssize\_t}.
\end{itemize}

There are probably other similar API changes required.

\section{What This TIP Does Not Do}

This TIP does not rearrange structure orderings. Although this would be very useful for some common structures (notably \textit{Tcl\_Obj}) if the common arithmetic types were smaller than the word size, it turns out that the changes in types required to deal with larger entities will make these rearrangements largely unnecessary and/or pointless. (Inefficiency in statically-allocated structures won't matter as the number of instances will remain comparatively small, even in very large programs.) Once the changes are applied, there is typically at most a single \textit{int} field per structure, usually holding either a reference count, a set of flags, or a Tcl result code.

It should also be noted that all structures are always going to be correctly aligned internally as we never use C's bitfield support, so structure alignment is purely an issue of efficiency, and not of correct access to the fields.

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\end{document}
