go-pcre2-lite

English | 简体中文

A high-performance, embeddable regular-expression library for Go, built on a trimmed PCRE2 8-bit interpreter compiled from vendored C source via cgo — JIT permanently disabled — and shipped as a drop-in replacement for github.com/dlclark/regexp2.

• Drop-in regexp2 compatibility — usually just change the import path.
• Faster, lower-allocation — faster than dlclark/regexp2 on every benchmark (up to ~9.6x), zero-allocation boolean matching, and a batched FindAll that beats Go's own regexp on massive small-match workloads (see Performance).
• ReDoS-safe by default — bounded by the PCRE2 match/depth limits; a catastrophic pattern returns an error instead of hanging.
• Rich feature set — capture/named groups, lookahead/lookbehind, backreferences, atomic groups, possessive quantifiers, recursion, \K, subroutine calls, Unicode properties.
• SQLite-style embedding — no external libs, no pkg-config, no CMake, no dynamic linking. CGO_ENABLED=1 is the only requirement.

Install

go get github.com/VillanCh/go-pcre2-lite

Requires CGO_ENABLED=1 and a C toolchain (the PCRE2 C source is vendored and compiled with the package; JIT is never referenced).

The repository and published module have no dependency on github.com/dlclark/regexp2. Compatibility is pinned with self-contained golden tests, PCRE2's official corpus, and downstream Goja Test262 runs.

Usage

Drop-in regexp2 replacement (rune-oriented, recommended)

import regexp2 "github.com/VillanCh/go-pcre2-lite/regexp2"

re := regexp2.MustCompile(`(?<area>\d{3})-(?<num>\d{4})`, 0)

// Boolean match (allocation-free hot path).
ok, _ := re.MatchString("call 555-1234")

// First match + named/numbered groups.
m, _ := re.FindStringMatch("call 555-1234")
if m != nil {
    fmt.Println(m.GroupByName("area").String()) // 555
    fmt.Println(m.GroupByNumber(2).String())    // 1234
    fmt.Println(m.Index, m.Length)              // rune index / length
}

// Iterate every match.
for m, _ := re.FindStringMatch("555-1234 / 777-0000"); m != nil; m, _ = re.FindNextMatch(m) {
    fmt.Println(m.String())
}

// Replace ($1 / ${name} templates) and ReplaceFunc.
out, _ := re.Replace("555-1234", "${area}.${num}", -1, -1) // 555.1234
out, _ = re.ReplaceFunc("555-1234", func(m regexp2.Match) string {
    return strings.ReplaceAll(m.String(), "-", " ")
}, -1, -1)

// Escape / Unescape literal text.
lit := regexp2.Escape("a.b*c")

// Defend against adversarial patterns (extension beyond the regexp2 API).
_ = re.SetMatchLimits(100000, 100000)

Index/Length are rune indices, exactly like regexp2. No Close() is needed (a finalizer reclaims the C memory), matching regexp2's ergonomics. A compiled *Regexp is safe for concurrent matching.

Low-level byte API (maximum performance)

import pcre2 "github.com/VillanCh/go-pcre2-lite"

re := pcre2.MustCompile(`\w+@\w+\.\w+`, pcre2.CompileOptions{UTF: true, UCP: true})
defer re.Close() // explicit release; the finalizer is a safety net

ok, _ := re.Match([]byte("a@b.com"))           // allocation-free
m, _ := re.Find([]byte("a@b.com"), 0)          // m.Groups[i] are byte spans
all, _ := re.FindAll([]byte("a@b.com x@y.io"), -1)

All offsets in the low-level API are UTF-8 byte offsets (vs. rune indices in the compat layer). Use this layer for the lowest allocation and the fastest batch FindAll.

Migration from github.com/dlclark/regexp2

For the overwhelming majority of code, migration is a one-line import change:

// before
import "github.com/dlclark/regexp2"
// after
import regexp2 "github.com/VillanCh/go-pcre2-lite/regexp2"

The exported surface (types, methods, RegexOptions constants, rune-based Index/Length, Replace/ReplaceFunc, Escape/Unescape) mirrors regexp2. Whole-match results agree on 100% of the 1585-input PCRE2 official corpus. The behavioural differences to be aware of:

Area	dlclark/regexp2 (.NET)	go-pcre2-lite
ReDoS / timeout	MatchTimeout = forever by default; can hang	bounded by match/depth limit; returns ErrMatchLimit instead of hanging
MatchTimeout field	enforces a wall-clock abort	accepted for API compat, not enforced — use SetMatchLimits
Mixed named + unnamed group numbering	unnamed first, then named	strict left-to-right
Repeated-group Group.Captures	full capture history	final capture only (.String() is identical)
RightToLeft	real right-to-left scan	accepted but engine always scans left-to-right

Only group numbering with mixed named/unnamed groups differs; access by name (GroupByName) is always identical. See MIGRATION.md for the full details and per-difference tests.

Unsupported / divergent syntax

These are the constructs to check when porting patterns. They fall into three buckets; the silent ones are the dangerous category.

Rejected at compile time (safe — you find out immediately)

Construct	Example	Note
.NET balancing groups	(?<open>\()[^()]*(?<-open>\))	.NET-only stack feature; PCRE2 has no equivalent
Lookbehind length depending on a backreference without ECMAScript mode	(?<=a(.\2)b(\1))	raw PCRE2 cannot bound it; ECMAScript mode handles it with the compatibility executor
Long \p{...} category names	\p{Number}, \p{IsGreek}	use the short alias \p{N}, \p{Greek} (also rejected by dlclark)

Accepted via the compatibility layer (regexp2 package)

The drop-in regexp2 package rewrites a few constructs that .NET/RE2 tolerate but raw PCRE2 rejects, so the common case "just works":

Construct	Example	Handling
ECMAScript lookbehind	(?<=a+)b, (?<=\1(\w))d, (?<=a(.\2)b(\1))	in ECMAScript mode, the main match and reversed assertion are both executed by PCRE2; this preserves JavaScript's right-to-left captures, alternatives, nested assertions, and backreferences without a 512-character cap
Non-ECMAScript variable lookbehind	(?<=a+)b, (?<="text":\s*")	the .NET-compat path retains the existing {n,512} bound for otherwise-unbounded quantifiers
Set shorthand beside - in a class	[\d\w-_], [a-\w]	the - is treated as a literal (as .NET/RE2 do), avoiding "invalid range in character class"
ECMAScript whitespace / dot	\s, \S, ., [\Sx]	exact JavaScript WhiteSpace + LineTerminator semantics, including U+2028/U+2029 and BOM
ECMAScript empty classes	[], [^]	never-match / match-any-character semantics, including line terminators
ECMAScript escapes	\x0, \u0065, \u{65}	legacy identity escapes and Unicode code-point escapes are translated to PCRE2 equivalents
ECMAScript unset references	\1(A), `(A	(B))\2`
ECMAScript capture names	(?<$>a), (?<π>b), escaped astral names	names are validated as IdentifierNames, mapped to safe PCRE2 names, and restored by the public Group API

Silently different (compiles, but behaves differently — audit these)

Construct	Example	dlclark (.NET)	go-pcre2-lite (PCRE2)
.NET character-class subtraction	[a-z-[aeiou]]	"set minus": matches b, not e	parsed as the class [a-z\-\[aeiou] followed by a literal ]; b alone does not match
Quantified capture in lookbehind without ECMAScript mode	(?<=(\w){3})def	group 1 = "a"	raw PCRE2 semantics return group 1 = "c"; use ECMAScript for JavaScript semantics
Backreference inside lookbehind without ECMAScript mode	(?<=\1(\w))d	matches	raw PCRE2 semantics diverge; use ECMAScript

Supported here, but NOT by dlclark/regexp2 (bonus PCRE2 power)

Possessive quantifiers a++, atomic groups (?>…), recursion (?R), \K, and subroutine calls (?&name) all compile here and are rejected by dlclark. Patterns relying on these are not portable back to regexp2.

JavaScript runtimes: keep a narrow UTF-16 fallback

PCRE2's 8-bit UTF mode cannot represent isolated UTF-16 surrogate code units (U+D800-U+DFFF). The rune-oriented APIs therefore return the exported ErrUnsupportedRune instead of silently converting such input to U+FFFD. JavaScript runtimes such as Goja must explicitly handle a pattern containing lone-surrogate escapes, or an input that returns ErrUnsupportedRune. A runtime can reject it or use a reversible reserved-code-point mapping while preserving UTF-16 indexes; silently converting it to U+FFFD is incorrect.

Safety: catastrophic backtracking is bounded

Unlike dlclark/regexp2 (whose default MatchTimeout is "forever"), every match here is bounded by the PCRE2 match/depth limits. A classic exponential ReDoS such as (a+)+$ against "aaaa…!" returns ErrMatchLimit in ~120 ms at the default limit, and in ~0.6 ms with SetMatchLimits(50000, …) — it never hangs and never overflows the stack (PCRE2 10.x matches on the heap).

The library is tested against real-world JS-ecosystem ReDoS CVEs (moment.js CVE-2022-31129, Cloudflare-2019, CWE-1333, UAParser.js CVE-2020-7733): all terminate. Note one caveat — the match limit bounds exponential backtracking but not a polynomial (e.g. quadratic) scan; for polynomial patterns the effective defense is capping input length.

Performance

Historical migration baseline measured with go test -bench, darwin/arm64 (Apple M-series), before the comparison engine was removed from the repository. Three backends ran identical work: dlclark = the engine we replace, drop-in = this regexp2 compat layer (rune output), low-level = the byte API. std = the Go standard library regexp (RE2), shown where its syntax allows. These figures are retained as historical migration evidence; current CI benchmarks are self-contained and do not import the comparison engine.

The drop-in layer is faster than dlclark/regexp2 on every benchmark, and the low-level byte API is faster still (1.6x–9.6x). The cgo match path also got notably faster with the PCRE2 10.47 upgrade (boolean matching dropped from ~1100 ns to ~680 ns while the pure-Go dlclark control was unchanged).

Scenario	dlclark	drop-in	low-level	speedup	drop-in alloc
Boolean match, short string	6472 ns	676 ns	674 ns	9.6x	0 B / 0
Boolean match, 100 KB input	26.4 ms	2.84 ms	2.84 ms	9.3x	0 B / 0
Match w/ backreference	396 ns	186 ns	185 ns	2.1x	0 B / 0
Backtracking-heavy, 32 KB	20.0 ms	10.9 ms	11.0 ms	1.8x	0 B / 0
Unicode \p{Han}, 8 KB	15.2 µs	12.2 µs	4.95 µs	1.2x	0 B / 0
Find with 6 captures	1072 ns	984 ns	690 ns	1.1x	752 B / 7
Compile (complex pattern)	10.3 µs	~3.2 µs	3.16 µs	3.3x	1.5 KB / 17
Find-all, 670 matches	380 µs	138 µs	63 µs	2.8x (ll 6.0x)	193 KB / 2004
Find-all, 30k matches	6.06 ms	5.24 ms	2.88 ms	1.2x (ll 2.1x)	7.8 MB / 90k

Boolean matching is allocation-free on the hot path (0 B / 0 allocs).

Optimizing massive numbers of small matches

Iterating very many tiny matches over a large subject used to be the one place the engine lost to pure-Go backtrackers. The cause was not the cgo boundary but O(n²) UTF-8 validation: with PCRE2_UTF set, PCRE2 re-validates the whole subject on every pcre2_match call, so N matches over an N-byte subject cost O(N²).

Two changes fixed it:

1. Batched FindAll/iteration — a single C function (p2l_match_all) gathers a chunk of matches per cgo call, turning N round trips into ⌈N/256⌉ and decoding one backing slice per chunk (low-level FindAll allocations dropped from 676 to 14 for 670 matches).
2. Validate-once — the batched loop validates UTF-8 on its first match and then sets PCRE2_NO_UTF_CHECK, collapsing the O(N²) revalidation to O(N).

The combined effect on 30,000 tiny matches over 30 KB: the low-level API went from 171 ms to 2.9 ms (≈59x) and the drop-in layer from 170 ms to 5.2 ms, both now faster than dlclark (6.1 ms) and Go's regexp (5.5 ms). Per-match iteration also uses the official pcre2_next_match() helper added in PCRE2 10.47, replacing the previously hand-rolled empty-match advancement.

The batched path also slashes heap allocations, which matters for GC pressure in long-running services:

ReDoS cost

(a+)+$ on a 40-char adversarial input:

Limit	Result
dlclark, no timeout	hangs (catastrophic)
default match limit	ErrMatchLimit in ~120 ms (bounded)
SetMatchLimits(50000, …)	ErrMatchLimit in ~0.6 ms

Run the current dependency-free benchmark suite with:

CGO_ENABLED=1 go test -bench . -benchmem -run '^$' ./...

Compatibility verification

• Migration API goldens: self-contained tests pin replacement expansion, full iteration, rune offsets, named/numbered group access, unset captures, escaping, and start-offset behavior without importing another regex engine.
• PCRE2 10.47 ground truth: match results agree on 929/931 (8-bit) and 1502/1504 (UTF) cases from PCRE2's own testoutput2/testoutput4 corpora; compile accept/reject agrees on 1258/1258 accepted and 385/388 rejected (the handful of misses are boundary cases the corpus parser approximates, not engine bugs).
• Per-version behaviour pins (pcre2_1047_regression_test.go): every behaviour-affecting changelog entry across 10.44–10.47 is asserted against the authoritative pcre2test golden output for that release — the variable-length lookbehind first-branch fix, \X ZWJ grapheme-cluster break, scan-substring assertions (*scs:) / (*scan_substring:), the (*ACCEPT)-inside-(*scs:) CVE-2025-58050 memory-safety fix, the new 10.47 subroutine-returning-captures (?N(group,...)), pcre2_next_match empty/\K-match iteration, UCD 16 properties, the raised 128-char group-name limit (boundary-tested at 128/129), and a guard that the 10.47 named-group hash-table lookup stays O(1).
• JavaScript/Node: ECMAScript test262 and V8 lookbehind/named-group/ Unicode-property cases, plus real-world ReDoS CVE safety.

Testing

CGO_ENABLED=1 go test ./...            # unit + golden + corpus + safety
CGO_ENABLED=1 go test -race ./...      # data-race free
CGO_ENABLED=1 go test -bench . ./...   # compatibility, low-level, stdlib, safety

Regenerating the vendored PCRE2 source

./tools/generate-pcre2lite/generate.sh   # download, trim, configure (no JIT)
./tools/verify-generated/verify.sh       # verify the committed sources are reproducible

License

The embedded PCRE2 C source is under the PCRE2 license (BSD-style); see THIRD_PARTY_LICENSES/PCRE2-LICENSE and the headers under internal/pcre2lite/.

A license for this project's own Go and wrapper code has not been chosen yet — add a LICENSE file before publishing.