[WIP] gh-129813, PEP 782: Add PyBytesWriter C API

[vstinner]

Add functions:

• PyBytesWriter_Create()
• PyBytesWriter_Discard()
• PyBytesWriter_Finish()
• PyBytesWriter_FinishWithSize()
• PyBytesWriter_FinishWithEndPointer()
• PyBytesWriter_Data()
• PyBytesWriter_Allocated()
• PyBytesWriter_SetSize()
• PyBytesWriter_Resize()

• Issue: [C API] PEP 782: Add PyBytesWriter API #129813

[vstinner]

This change has no impact on performance, even if the new public API allocates memory on the heap, instead of allocating on the stack. It uses a freelist to optimize PyBytesWriter_Create().

Microbenchmark on 3 functions, to compare the private _PyBytesWriter (ref) to the new public PyBytesWriter (change):

• bytes(list)
• bytes.fromhex(str)
• binascii.b2a_uu(bytes)

import pyperf
import binascii

runner = pyperf.Runner()
runner.bench_func('from list 100', bytes, list(b'x' * 100))
runner.bench_func('from list 1,000', bytes, list(b'x' * 1_000))

runner.bench_func('from hex 100', bytes.fromhex, bytes(range(100)).hex())
runner.bench_func('from hex 1,000', bytes.fromhex, (b'x' * 1_000).hex())

runner.bench_func('b2a_uu', binascii.b2a_uu, b'x' * 45)

Result:

Benchmark	ref	change
from list 100	631 ns	623 ns: 1.01x faster
from hex 100	141 ns	145 ns: 1.03x slower
from hex 1,000	1.03 us	1.04 us: 1.00x slower
b2a_uu	112 ns	111 ns: 1.01x faster
Geometric mean	(ref)	1.00x slower

Benchmark hidden because not significant (1): from list 1,000

[vstinner]

Benchmark comparing PyBytes_FromStringAndSize(NULL, length) (ref) to PyBytesWriter_Create() (change).

Benchmark:

import pyperf

SIZES = (10, 100, 500)

runner = pyperf.Runner()
for size in SIZES:
    large_int = (2 ** (size * 8) - 1)
    runner.bench_func(f'to_bytes({size})', large_int.to_bytes, size)
for size in SIZES:
    mem = memoryview(b'x' * size)
    runner.bench_func(f'memoryview({size}).tobytes()', mem.tobytes)

Result:

Benchmark	ref	change
to_bytes(10)	56.3 ns	66.4 ns: 1.18x slower (+10.1 ns)
to_bytes(100)	152 ns	162 ns: 1.06x slower (+10 ns)
to_bytes(500)	563 ns	559 ns: 1.01x faster (+4 ns)
memoryview(10).tobytes()	37.5 ns	47.0 ns: 1.25x slower (+9.5 ns)
memoryview(100).tobytes()	35.3 ns	46.6 ns: 1.32x slower (+11.3 ns)
memoryview(500).tobytes()	45.5 ns	55.3 ns: 1.21x slower (+9.8 ns)
Geometric mean	(ref)	1.16x slower

It's hard to beat PyBytes_FromStringAndSize(NULL, length) performance, since PyBytesWriter_Create() is a wrapper built on top of PyBytes_FromStringAndSize(NULL, length).

There is an overhead around 10 ns when using PyBytesWriter.