\documentclass[]{article}
\usepackage{amsmath,graphicx,supertabular,hyperref,tabularx,ifthen}
\title{TIP \#113: Multi-Line Searches in the Text Widget}
\date{October 11, 2002}
\author{Vince Darley}
\urlstyle{sf}
\setlength{\parskip}{1ex}
\setlength{\parindent}{0pt}
\def\tipversion$#1${\texttt{\$#1\$}}
\def\tiplangle#1{\ensuremath{<}}
\def\tiprangle#1{\ensuremath{>}}
\def\tipbar#1{\ensuremath{|}}
\def\tipmail#1#2{\(\langle\){\small\expandafter\url{#1@#2}}\(\rangle\)}
\ifx\pdfoutput\undefined
\newcommand{\tipimage}[2]{\typeout{Make sure you download #1.eps}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.eps}}{\includegraphics[width=0.8\textwidth]{#1.eps}}}
\newcommand{\tipxref}[1]{}
\newcommand{\tipxrefend}{}
\else
\newcommand{\tipimage}[2]{\typeout{Make sure you create #1.pdf}\ifthenelse{\lengthtest{0.8\textwidth>#2}\and\lengthtest{0pt<#2}}{\includegraphics{#1.pdf}}{\includegraphics[width=0.8\textwidth]{#1.pdf}}}
\pdfcatalog{/PageMode /UseOutlines}
\newcommand{\tipxref}[1]{\pdfannotlink attr {/C [0.5 0.5 1.0] /Border [0 0 1]} goto name {#1}}
\newcommand{\tipxrefend}{\pdfendlink}
\fi
\newenvironment{tipabstract}{\begin{abstract}}{\end{abstract}}
\begin{document}\maketitle
\begin{center}\begin{tabularx}{\linewidth}{|r@{: }X|}\hline
\textbf{TIP \#113}&\textbf{Multi-Line Searches in the Text Widget}\\\hline
Author&

Vince Darley \tipmail{vincentdarley}{users.sourceforge.net}
\\
Created&Friday, \(\text{11}^{\text{th}}\) October 2002\\
Type&Project\\
Tcl Version&\(8.5\)\\
State&Final\\
Vote&Done\\
Version&\tipversion$Revision: 1.13 $\\
Post History&\\
\hline\end{tabularx}\end{center}
\thispagestyle{empty}\pagestyle{empty}
\begin{tipabstract}

This TIP proposes enhancing the implementation of the \textit{\$textwidget search} subcommand to allow matching of both exact strings and regexp patterns which span multiple lines, and to allow reporting on all matches.
\end{tipabstract}
\tableofcontents\setcounter{page}{0}\clearpage\pagestyle{plain}

\section{Proposal}

If the string/pattern given to the \textit{search} subcommand contains sub-strings/patterns which match newlines, then it will be possible for that command to return a match which spans multiple lines. Where a match could occur both within a single line and across multiple lines, the first such match will be found, and the length of the match will follow the usual regexp rules, as documented in the regexp man page. Since the text widget is inherently a line-based container of text, regexp searches will implicitly use the regexp \textit{-line} functionality so that \textit{\textasciicircum{}}, \textit{\$} matching beginning and end of any line, and \textit{.}, \textit{[\textasciicircum{}} will not match a newline (but see the \textit{-nolinestop} flag to turn off the latter behaviour).

This can be implemented very efficiently, given the \textit{TCL\_REG\_CANMATCH} flag supported by the regexp library, with no impact at all on the speed of matching single lines.

In addition, two new options to the \textit{search} subcommand are available:

If the new \textit{-all} option is given to the \textit{search} subcommand, then all matches within the given range will be reported. This means the return result of the command will be a list of indices, and, if a \textit{-count var} option was given, \textit{var} will be set to a list of match-index-lengths.

If the new \textit{-nolinestop} option is given then regexp searches will allow \textit{.} and \textit{[\textasciicircum{}} sequences to match newline characters (which is normally not the case). This is equivalent to \textit{not} providing the \textit{-linestop} flag to Tcl's \textit{regexp} command.

The text widget man page will be updated to reflect the new \textit{-all} and \textit{-nolinestop} options, and to remove the ``single line'' caveat.

\section{Reference implementation}

This is available from:

\url{http://sourceforge.net/tracker/?func=detail&aid=621901&group_id=12997&atid=312997}

The patch includes objectification of the entire Text widget, so the multi-line search changes are not obvious to isolate. In fact the changes required are \tiplangle{} 100 lines of code (given that the rest has been objectified, that is). Of course one nice side-effect of objectification is that regexp objects used in searches are actually cached, which they previously couldn't be.

Note: this patch has to workaround a crashing bug in Tcl's unicode string manipulation. It would be best if that bug was fixed before applying this patch.

\section{Issues}

On the implementation side, it might be interesting to abstract the search interface away from the text widget, so that it could in principle be applied to any line-based textual source.

As in the single-line matching implementation in Tcl 8.x, the lack of support for backwards matching in Tcl's regexp library means that backwards matching can only be implemented as repeated forward matches, with a commensurate performance penalty (the solution to which is outside the scope of this tip).

Tk has a curious misfeature that \textit{\$text search -regexp ``\(\backslash\)n'' \$pos} will always fail to match anything. This behaviour will change as a result of this TIP (it will match the first newline after \textit{\$pos}), and any code which somehow depended on that peculiarity will therefore break.

\section{Copyright}

This document has been placed in the public domain.

\section{Colophon}

\textit{TIP AutoGenerator --- written by Donal K. Fellows}
\end{document}
