Skip to content

Free-Threaded Python 3.14 (no-GIL) for Plone: Feasibility Report #4277

@jensens

Description

@jensens

TL;DR

Plone cannot currently run on free-threaded Python 3.14. Three core Zope Foundation C extensions fail to even compile, which cascades to block the entire ZODB/Zope stack. Even if those were fixed, lxml would re-enable the GIL anyway.

Report

Date: 2026-02-16
Environment: Plone 6.2.0a2.dev0, Python 3.14.0, coredev (mxdev/mxmake)
Free-threaded Python: cpython-3.14.0+freethreaded via uv python install 3.14t

Executive Summary

Plone cannot currently run on free-threaded Python 3.14t. Three Zope Foundation C extension packages fail to compile, which cascades to block the entire ZODB/Zope persistence and security stack. Even after fixing compilation, lxml (used pervasively in Plone) re-enables the GIL on import, negating any free-threading benefit.

This report categorizes the work needed into phases, from quick mechanical fixes to deep architectural changes.


Phase 1: Compilation Fixes (upstream contributions)

These packages fail to build on python3.14t because they directly access PyObject->ob_refcnt, which is no longer a simple struct field in the free-threaded build (uses atomic refcounting instead). The fix is mechanical: replace obj->ob_refcnt with Py_REFCNT(obj) / Py_SET_REFCNT(obj, n).

1.1 persistent (v6.5)

  • Repo: https://github.com/zopefoundation/persistent
  • File: src/persistent/cPickleCache.c
  • Issue: 5 direct ob_refcnt accesses (lines 388, 531, 532, 538, 541)
  • Note: PR #214 already fixed cPersistence.c but missed cPickleCache.c
  • Fix:
    • Line 388: v->ob_refcnt <= 1 -> Py_REFCNT(v) <= 1
    • Line 531: v->ob_refcnt <= 0 -> Py_REFCNT(v) <= 0
    • Lines 532, 538, 541: v->ob_refcnt -> Py_REFCNT(v) in Py_BuildValue calls
  • Effort: Small - single PR
  • Unblocks: BTrees, ZODB, zope.container (and everything above)

1.2 ExtensionClass (v6.2)

  • Repo: https://github.com/zopefoundation/ExtensionClass
  • File: src/ExtensionClass/_ExtensionClass.c
  • Issue: 1 direct ob_refcnt access (line 887)
  • Note: PR #81 added "preliminary 3.14 support" but did not address the free-threaded build
  • Fix:
    • Line 887: callable->ob_refcnt == 1 -> Py_REFCNT(callable) == 1
  • Effort: Trivial - single line
  • Unblocks: Acquisition, AccessControl, Persistence (and everything above)

1.3 zodbpickle (v4.3)

  • Repo: https://github.com/zopefoundation/zodbpickle
  • File: src/zodbpickle/_pickle_33.c
  • Issues:
    1. ob_refcnt direct access (1 occurrence)
    2. PyFloat_Pack8 / PyFloat_Unpack8 type signature changed from unsigned char * to char * in Python 3.14
  • Note: PR #104 added 3.14 support but did not cover the free-threaded build
  • Fix: Update signatures and replace ob_refcnt access
  • Effort: Small - single PR
  • Unblocks: ZODB

Cascade: Packages unblocked by Phase 1

Once the above three are fixed, these packages (which failed only due to transitive dependencies) should install:

Package Version Blocked by
BTrees 6.3 persistent
Acquisition 6.2 ExtensionClass
AccessControl 7.3 ExtensionClass
Persistence 5.4 ExtensionClass
ZODB 6.1 persistent, BTrees
zope.container 7.2 persistent

Phase 2: Packages That Already Work

These were verified to compile and install on python3.14t:

Package Version Notes
zope.interface 8.2 Builds from source, OK
zope.proxy 7.1 Builds from source, OK
zope.security 8.3 Builds from source, OK
zope.hookable 8.2 Builds from source, OK
zope.i18nmessageid 8.2 Builds from source, OK
Pillow 12.1.1 cp314t wheels on PyPI, CI-tested, supported since 11.0.0
cffi 2.0.0 Full free-threaded support
cryptography 46.0.5 Full free-threaded support (Rust/PyO3)
PyYAML 6.0.3 cp314t wheels on PyPI (not CI-tested)
wrapt 2.1.1 Full free-threaded support since 1.17.0
MarkupSafe 3.0.3 Builds from source, OK
Chameleon 4.6.0 Pure Python
RestrictedPython 8.1 Pure Python

Phase 3: lxml - The Fundamental Blocker

The Problem

lxml's Cython/C code fundamentally relies on the GIL for thread safety. From the maintainer (Stefan Behnel):

"lxml makes tight use of the GIL for performance reasons, and the _elementFactory function is central, critical and carefully crafted to benefit from the GIL without requiring additional locking."

The test suite segfaults under free-threaded Python in multi-threaded tests (test_concurrent_class_lookup, test_concurrent_proxies).

Impact on Plone

lxml is used everywhere in Plone:

  • Chameleon TAL template compilation (HTML/XML parsing)
  • Products.PortalTransforms (content transformations)
  • plone.outputfilters (output processing)
  • diazo / plone.app.theming (XSLT theme transforms)
  • plone.namedfile (SVG handling)
  • Content import/export

Even if all Zope packages were fixed, importing lxml re-enables the GIL, negating all free-threading benefits.

Action

  • Monitor lxml's Launchpad bug and GitHub for progress
  • No Plone-side workaround exists - lxml is deeply embedded in the stack
  • Consider contributing to lxml's free-threading effort if resources allow

Phase 4: Runtime Thread-Safety (Long-Term)

Even after compilation fixes (Phase 1) and lxml support (Phase 3), the Zope/ZODB stack was designed with the GIL as an implicit lock. True free-threaded operation requires thread-safety audits and fixes across the stack.

4.1 ZODB / persistent

  • Pickle cache (cPickleCache.c): Ghost/active object state transitions are not thread-safe without the GIL
  • Connection management: Connection._registered_objects, _added, _modified are plain dicts/lists
  • Transaction machinery: Global transaction manager state
  • BTrees: Not safe for concurrent writes; concurrent reads may also be unsafe without GIL

4.2 Acquisition

  • Implicit acquisition wrapping creates complex object graphs at runtime
  • aq_acquire, aq_parent traversal assumes stable object references
  • Would need locking or copy-on-write semantics

4.3 AccessControl

  • Security checks use global/thread-local state (SecurityManager)
  • Permission caching assumes GIL protection
  • guarded_import already has a known race condition (see zopefoundation/AccessControl PR "Prevent race condition in guarded_import")

4.4 Chameleon (pure Python concerns)

  • Template compilation cache (_template_cache) is a plain dict
  • Concurrent compilation of the same template could cause issues
  • Would need threading.Lock around cache access

4.5 Zope Foundation's Position

In zopefoundation/meta#240 (May 2024), the maintainer stated:

"I have no idea how a free-threaded version would interact with any of our packages."

The free-threaded build is currently skipped in CI for Zope Foundation packages. There is no active initiative to support free-threaded Python.


Phase 5: Test Infrastructure

  • grpcio (used by robotframework-browser via plone.app.robotframework): No cp314t wheels, no free-threaded support. Robot/acceptance tests won't run.
  • Action: Not a runtime blocker. Robot tests would need to wait for grpcio free-threaded support or use an alternative test approach.

Summary: Action Items by Priority

Immediate / Low-Effort (contribute upstream)

# Action Target Repo Effort
1 Fix ob_refcnt in cPickleCache.c zopefoundation/persistent ~1h
2 Fix ob_refcnt in _ExtensionClass.c zopefoundation/ExtensionClass ~15min
3 Fix build issues in _pickle_33.c zopefoundation/zodbpickle ~2h

These are good contributions regardless of free-threading - they fix forward compatibility with CPython internal API changes.

Medium-Term / Monitor

# Action Who
4 Monitor lxml free-threading progress lxml maintainers
5 Request Zope Foundation CI for free-threaded builds zopefoundation/meta
6 Audit Chameleon template cache for thread safety Chameleon maintainers

Long-Term / Major Effort

# Action Scope
7 Thread-safety audit of ZODB connection/cache/transaction Months of work
8 Thread-safety audit of BTrees for concurrent access Significant
9 Thread-safety audit of Acquisition wrapping Significant
10 Thread-safety audit of AccessControl security checks Significant

Appendix: Test Environment Details

Python (standard):  3.14.0 (main, Oct 14 2025) [Clang 20.1.4]
Python (free-threaded): 3.14.0 free-threading build (main, Oct 28 2025) [Clang 20.1.4]
Plone: 6.2.0a2.dev0
Total packages: 350
C extensions: ~70 .so files
Build system: mxdev + mxmake + uv

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions