asyncio.StreamReader raises BufferError with active memoryview

[MadLongTom]

Bug report

Bug description:

Summary

When StreamReader has an active memoryview export of its internal buffer, methods that mutate the buffer in place can raise:

BufferError: Existing exports of data: object cannot be re-sized

This appears in real workloads with async database drivers, but the root cause is in StreamReader buffer mutation behavior.

Steps to reproduce

import asyncio

async def main():
    r = asyncio.StreamReader()
    r.feed_data(b"AABBCCDD")

    # Simulate a consumer that keeps a memoryview alive
    mv = memoryview(r._buffer)

    # Triggers BufferError in current implementation
    await r.readexactly(4)

    mv.release()

asyncio.run(main())

Expected behavior

readexactly(4) should return b"AABB" without raising BufferError.

Actual behavior

BufferError: Existing exports of data: object cannot be re-sized

Why this happens

Some StreamReader paths mutate bytearray in place, for example slice deletion and clear operations. In-place resize is not allowed while a memoryview export exists.

Affected methods

• read
• readline
• readuntil
• readexactly

Environment

• Python: 3.13.x, 3.14.x (also reproducible in local tests with newer builds)
• OS: macOS (also observed in Linux server workloads)
• Third-party context where this is frequently hit: async DB drivers using memoryview over packet buffers

Possible fix direction

Replace in-place buffer mutations with non-mutating replacement assignments, for example:

• replace slice delete with reassignment to remaining bytes
• replace clear with assigning a new empty bytearray

If useful, I can open a PR with tests covering the regression paths.

CPython versions tested on:

3.13

Operating systems tested on:

macOS

Linked PRs

• gh-146379: Avoid StreamReader BufferError with active memoryview exports #146380

[kumaraditya303]

When StreamReader has an active memoryview export of its internal buffer, methods that mutate the buffer in place can raise:

The internal buffer is private, it can only have a nonzero export count when the buffer is accessed directly, or is there any other way it happens?

The buffer is private because we don't want users to access it.

[MadLongTom]

When StreamReader has an active memoryview export of its internal buffer, methods that mutate the buffer in place can raise:

The internal buffer is private, it can only have a nonzero export count when the buffer is accessed directly, or is there any other way it happens?

The buffer is private because we don't want users to access it.

You're right that _buffer is private. However, the BufferError can be triggered purely through CPython's own code path in readexactly (Python 3.14.2):

# streams.py line ~774-776 (3.14.2)
data = bytes(memoryview(self._buffer)[:n])
del self._buffer[:n]

In a single-expression evaluation, CPython's refcounting releases the temporary memoryview before the del line runs. But under certain async conditions — specifically when a previous coroutine frame or traceback holds a reference to a local that transitively pins a memoryview on the buffer — the export count remains nonzero when del self._buffer[:n] executes.

In our production workload we use asyncmy (a Cython-based async MySQL driver) with SQLAlchemy async. asyncmy calls reader.readexactly(n) in a tight loop to parse MySQL wire protocol packets (read_packet → _read_bytes → readexactly). The driver never accesses _buffer directly; it only calls the public readexactly API. The BufferError is triggered internally by readexactly's own memoryview(self._buffer) temporary when coroutine frame cleanup is delayed.

The main branch already migrated most callsites to take_bytes(), but readline's LimitOverrunError handler still uses del self._buffer[:n] and .clear(). And take_bytes() itself also calls _canresize() internally, so it has the same BufferError risk when an export exists.