Overview

Package syntax parses regular expressions into parse trees and compiles parse
trees into programs. Most clients of regular expressions will use the facilities
of package regexp (such as Compile and Match) instead of this package.

Syntax

The regular expression syntax understood by this package when parsing with the
Perl flag is as follows. Parts of the syntax can be disabled by passing
alternate flags to Parse.

Single characters:

Composites:

  1. xy             x followed by y
  2. x|y            x or y (prefer x)

Repetitions:

  1. x*             zero or more x, prefer more
  2. x+             one or more x, prefer more
  3. x?             zero or one x, prefer one
  4. x{n,m}         n or n+1 or ... or m x, prefer more
  5. x{n,}          n or more x, prefer more
  6. x{n}           exactly n x
  7. x*?            zero or more x, prefer fewer
  8. x+?            one or more x, prefer fewer
  9. x??            zero or one x, prefer zero
  10. x{n,m}?        n or n+1 or ... or m x, prefer fewer
  11. x{n,}?         n or more x, prefer fewer
  12. x{n}?          exactly n x

Implementation restriction: The counting forms x{n,m}, x{n,}, and x{n} reject
forms that create a minimum or maximum repetition count above 1000. Unlimited
repetitions are not subject to this restriction.

Grouping:

  1. (re)           numbered capturing group (submatch)
  2. (?P<name>re)   named & numbered capturing group (submatch)
  3. (?:re)         non-capturing group
  4. (?flags)       set flags within current group; non-capturing
  5. (?flags:re)    set flags during re; non-capturing
  7. Flag syntax is xyz (set) or -xyz (clear) or xy-z (set xy, clear z). The flags are:
  9. i              case-insensitive (default false)
  10. m              multi-line mode: ^ and $ match begin/end line in addition to begin/end text (default false)
  11. s              let . match \n (default false)

Empty strings:

  1. ^              at beginning of text or line (flag m=true)
  2. $              at end of text (like \z not Perl's \Z) or line (flag m=true)
  3. \A             at beginning of text
  4. \b             at ASCII word boundary (\w on one side and \W, \A, or \z on the other)
  5. \B             not at ASCII word boundary
  6. \z             at end of text

Escape sequences:

  1. \a             bell (== \007)
  2. \f             form feed (== \014)
  3. \t             horizontal tab (== \011)
  4. \n             newline (== \012)
  5. \r             carriage return (== \015)
  6. \v             vertical tab character (== \013)
  7. \*             literal *, for any punctuation character *
  8. \123           octal character code (up to three digits)
  9. \x7F           hex character code (exactly two digits)
  10. \x{10FFFF}     hex character code
  11. \Q...\E        literal text ... even if ... has punctuation

Character class elements:

  1. x              single character
  2. A-Z            character range (inclusive)
  3. \d             Perl character class
  4. [:foo:]        ASCII character class foo
  5. \p{Foo}        Unicode character class Foo
  6. \pF            Unicode character class F (one-letter name)

Named character classes as character class elements:

  1. [\d]           digits (== \d)
  2. [^\d]          not digits (== \D)
  4. [^\D]          not not digits (== \d)
  5. [[:name:]]     named ASCII class inside character class (== [:name:])
  6. [^[:name:]]    named ASCII class inside negated character class (== [:^name:])
  7. [\p{Name}]     named Unicode property inside character class (== \p{Name})
  8. [^\p{Name}]    named Unicode property inside negated character class (== \P{Name})

Perl character classes (all ASCII-only):

  1. \d             digits (== [0-9])
  2. \D             not digits (== [^0-9])
  3. \s             whitespace (== [\t\n\f\r ])
  4. \S             not whitespace (== [^\t\n\f\r ])
  5. \w             word characters (== [0-9A-Za-z_])
  6. \W             not word characters (== [^0-9A-Za-z_])

ASCII character classes:

  1. [[:alnum:]]    alphanumeric (== [0-9A-Za-z])
  2. [[:alpha:]]    alphabetic (== [A-Za-z])
  3. [[:ascii:]]    ASCII (== [\x00-\x7F])
  4. [[:blank:]]    blank (== [\t ])
  5. [[:cntrl:]]    control (== [\x00-\x1F\x7F])
  6. [[:digit:]]    digits (== [0-9])
  7. [[:graph:]]    graphical (== [!-~] == [A-Za-z0-9!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~])
  8. [[:lower:]]    lower case (== [a-z])
  9. [[:print:]]    printable (== [ -~] == [ [:graph:]])
  10. [[:punct:]]    punctuation (== [!-/:-@[-`{-~])
  11. [[:upper:]]    upper case (== [A-Z])
  12. [[:word:]]     word characters (== [0-9A-Za-z_])
  13. [[:xdigit:]]   hex digit (== [0-9A-Fa-f])

Index

compile.go parse.go prog.go simplify.go

func 

  1. func IsWordChar(r ) bool

IsWordChar reports whether r is consider a ``word character’’ during the
evaluation of the \b and \B zero-width assertions. These assertions are
ASCII-only: the word characters are [A-Za-z0-9_].

  1. type EmptyOp

An EmptyOp specifies a kind or mixture of zero-width assertions.

  1. const (
  2.     EmptyBeginLine EmptyOp = 1 << 
  3.     EmptyEndLine
  4.     EmptyBeginText
  5.     EmptyEndText
  6.     EmptyWordBoundary
  7.     EmptyNoWordBoundary
  8. )

func EmptyOpContext

  1. func EmptyOpContext(r1, r2 rune)

EmptyOpContext returns the zero-width assertions satisfied at the position
between the runes r1 and r2. Passing r1 == -1 indicates that the position is at
the beginning of the text. Passing r2 == -1 indicates that the position is at
the end of the text.

type Error

An Error describes a failure to parse a regular expression and gives the
offending expression.

func (*Error) Error

  1. func (e *Error) Error()

type ErrorCode

  1. type ErrorCode string

An ErrorCode describes a failure to parse a regular expression.

  1. const (
  2.     // Unexpected error
  3.     ErrInternalError  = "regexp/syntax: internal error"
  4.  
  5.     // Parse errors
  6.     ErrInvalidCharClass      ErrorCode = "invalid character class"
  7.     ErrInvalidCharRange       = "invalid character class range"
  8.     ErrInvalidEscape         ErrorCode = "invalid escape sequence"
  9.     ErrInvalidNamedCapture    = "invalid named capture"
  10.     ErrInvalidPerlOp         ErrorCode = "invalid or unsupported Perl syntax"
  11.     ErrInvalidRepeatOp        = "invalid nested repetition operator"
  12.     ErrInvalidRepeatSize     ErrorCode = "invalid repeat count"
  13.     ErrInvalidUTF8            = "invalid UTF-8"
  14.     ErrMissingBracket        ErrorCode = "missing closing ]"
  15.     ErrMissingParen           = "missing closing )"
  16.     ErrMissingRepeatArgument ErrorCode = "missing argument to repetition operator"
  17.     ErrTrailingBackslash      = "trailing backslash at end of expression"
  18.     ErrUnexpectedParen       ErrorCode = "unexpected )"
  19. )

func (ErrorCode) 

  1. func (e ) String() string

type 

  1. type Flags

Flags control the behavior of the parser and record information about regexp
context.

  1. const (
  2.     FoldCase      Flags = 1 <<  // case-insensitive match
  3.     Literal                         // treat pattern as literal string
  4.     ClassNL                         // allow character classes like [^a-z] and [[:space:]] to match newline
  5.     DotNL                           // allow . to match newline
  6.     OneLine                         // treat ^ and $ as only matching at beginning and end of text
  7.     NonGreedy                       // make repetition operators default to non-greedy
  8.     PerlX                           // allow Perl extensions
  9.     UnicodeGroups                   // allow \p{Han}, \P{Han} for Unicode group and negation
  10.     WasDollar                       // regexp OpEndText was $, not \z
  11.     Simple                          // regexp contains no counted repetition
  12.     MatchNL = ClassNL | 
  13.  
  14.     Perl        = ClassNL |  | PerlX |  // as close to Perl as possible
  15.     POSIX Flags = 0                                         // POSIX syntax
  16. )

  1. type Inst struct {
  2.     Op   
  3.     Out  uint32 // all but InstMatch, InstFail
  4.     Arg   // InstAlt, InstAltMatch, InstCapture, InstEmptyWidth
  5.     Rune []rune
  6. }

An Inst is a single instruction in a regular expression program.

func (*Inst) 

  1. func (i *) MatchEmptyWidth(before rune, after ) bool

MatchEmptyWidth reports whether the instruction matches an empty string between
the runes before and after. It should only be called when i.Op ==
InstEmptyWidth.

func (*Inst) 

  1. func (i *) MatchRune(r rune)

MatchRune reports whether the instruction matches (and consumes) r. It should
only be called when i.Op == InstRune.

  1. func (i *Inst) MatchRunePos(r ) int

MatchRunePos checks whether the instruction matches (and consumes) r. If so,
MatchRunePos returns the index of the matching rune pair (or, when len(i.Rune)
== 1, rune singleton). If not, MatchRunePos returns -1. MatchRunePos should only
be called when i.Op == InstRune.

func (*Inst) 

  1. func (i *) String() string

type 

  1. type InstOp

An InstOp is an instruction opcode.

  1. const (
  2.     InstAlt InstOp = 
  3.     InstAltMatch
  4.     InstCapture
  5.     InstEmptyWidth
  6.     InstMatch
  7.     InstFail
  8.     InstNop
  9.     InstRune
  10.     InstRune1
  11.     InstRuneAny
  12.     InstRuneAnyNotNL
  13. )

func (InstOp) String

type Op

  1. type Op uint8

An Op is a single regular expression operator.

  1. const (
  2.     OpNoMatch         = 1 + iota // matches no strings
  3.     OpEmptyMatch                   // matches empty string
  4.     OpLiteral                      // matches Runes sequence
  5.     OpCharClass                    // matches Runes interpreted as range pair list
  6.     OpAnyCharNotNL                 // matches any character except newline
  7.     OpAnyChar                      // matches any character
  8.     OpBeginLine                    // matches empty string at beginning of line
  9.     OpEndLine                      // matches empty string at end of line
  10.     OpBeginText                    // matches empty string at beginning of text
  11.     OpEndText                      // matches empty string at end of text
  12.     OpWordBoundary                 // matches word boundary `\b`
  13.     OpNoWordBoundary               // matches word non-boundary `\B`
  14.     OpCapture                      // capturing subexpression with index Cap, optional name Name
  15.     OpStar                         // matches Sub[0] zero or more times
  16.     OpPlus                         // matches Sub[0] one or more times
  17.     OpQuest                        // matches Sub[0] zero or one times
  18.     OpRepeat                       // matches Sub[0] at least Min times, at most Max (Max == -1 is no limit)
  19.     OpConcat                       // matches concatenation of Subs
  20.     OpAlternate                    // matches alternation of Subs
  21. )

type 

  1. type Prog struct {
  2.     Inst   []
  3.     Start  int // index of start instruction
  4.     NumCap  // number of InstCapture insts in re
  5. }

A Prog is a compiled regular expression program.

func Compile

  1. func Compile(re *Regexp) (*, error)

Compile compiles the regexp into a program to be executed. The regexp should
have been simplified already (returned from re.Simplify).

func (*Prog) 

  1. func (p *) Prefix() (prefix string, complete )

Prefix returns a literal string that all matches for the regexp must start with.
Complete is true if the prefix is the entire match.

func (*Prog) StartCond

  1. func (p *Prog) StartCond()

StartCond returns the leading empty-width conditions that must be true in any
match. It returns ^EmptyOp(0) if no matches are possible.

  1. func (p *Prog) String() 

  1. type Regexp struct {
  2.     Op       Op // operator
  3.     Flags    
  4.     Sub      []*Regexp  // subexpressions, if any
  5.     Sub0     [1]* // storage for short Sub
  6.     Rune     []rune     // matched runes, for OpLiteral, OpCharClass
  7.     Rune0    [2]    // storage for short Rune
  8.     Min, Max int        // min, max for OpRepeat
  9.     Cap              // capturing index, for OpCapture
  10.     Name     string     // capturing name, for OpCapture
  11. }

A Regexp is a node in a regular expression syntax tree.

func 

  1. func Parse(s , flags Flags) (*, error)

Parse parses a regular expression string s, controlled by the specified Flags,
and returns a regular expression parse tree. The syntax is described in the
top-level comment.

func (*Regexp) 

  1. func (re *) CapNames() []string

CapNames walks the regexp to find the names of capturing groups.

func (*Regexp) 

  1. func (x *) Equal(y *Regexp)

Equal returns true if x and y have identical structure.

func (*Regexp) MaxCap

  1. func (re *Regexp) MaxCap()

MaxCap walks the regexp to find the maximum capture index.

func (*Regexp) Simplify

  1. func (re *Regexp) Simplify() *

Simplify returns a regexp equivalent to re but without counted repetitions and
with various other simplifications, such as rewriting /(?:a+)+/ to /a+/. The
resulting regexp will execute correctly but its string representation will not
produce the same parse tree, because capturing parentheses may have been
duplicated or removed. For example, the simplified form for /(x){1,2}/ is
/(x)(x)?/ but both parentheses capture as $1. The returned regexp may share
structure with or be the original.