Skip to content

gh-146041: Avoid lock in sys.intern() for already interned strings#146072

Merged
colesbury merged 4 commits intopython:mainfrom
colesbury:gh-146041-intern-lock-free
Mar 24, 2026
Merged

gh-146041: Avoid lock in sys.intern() for already interned strings#146072
colesbury merged 4 commits intopython:mainfrom
colesbury:gh-146041-intern-lock-free

Conversation

@colesbury
Copy link
Contributor

@colesbury colesbury commented Mar 17, 2026

Fix free-threading scaling bottleneck in sys.intern and PyObject_SetAttr by avoiding the interpreter-wide lock when the string is already interned and immortalized.

@colesbury
Copy link
Contributor Author

@encukou, please let me know if you'd like me to do this differently. I've removed the assertion about unicode state in _Py_SetImmortalUntracked. It was awkward to keep because this PR requires that interned state is set to SSTATE_INTERNED_IMMORTAL as the last step in interning for thread-safety.

Copy link
Contributor

@mpage mpage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks correct to me

PyObject *r;
int res = PyDict_GetItemRef(interned, s, &r);
if (res < 0) {
PyErr_Clear();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearing the error here feels a little weird to me, but it's consistent with the rest of the function.

@colesbury colesbury added 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section needs backport to 3.14 bugs and security fixes labels Mar 24, 2026
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 6e59598 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F146072%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Mar 24, 2026
@colesbury colesbury merged commit 6009309 into python:main Mar 24, 2026
98 of 103 checks passed
@miss-islington-app
Copy link

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Sorry, @colesbury, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 60093096ba62110151d822b072a01061876e9404 3.14

@colesbury colesbury deleted the gh-146041-intern-lock-free branch March 24, 2026 18:29
colesbury added a commit to colesbury/cpython that referenced this pull request Mar 24, 2026
…ed strings (pythongh-146072)

Fix free-threading scaling bottleneck in sys.intern and `PyObject_SetAttr` by
avoiding the interpreter-wide lock when the string is already interned and
immortalized.
(cherry picked from commit 6009309)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Mar 24, 2026

GH-146390 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Mar 24, 2026
@colesbury colesbury removed their assignment Mar 24, 2026
colesbury added a commit that referenced this pull request Mar 25, 2026
…ings (gh-146072) (#146390)

Fix free-threading scaling bottleneck in sys.intern and `PyObject_SetAttr` by
avoiding the interpreter-wide lock when the string is already interned and
immortalized.

(cherry picked from commit 6009309)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants