Skip to content

[mypyc] Add librt.threading.Lock class#21690

Merged
JukkaL merged 45 commits into
masterfrom
librt-threading-lock
Jul 7, 2026
Merged

[mypyc] Add librt.threading.Lock class#21690
JukkaL merged 45 commits into
masterfrom
librt-threading-lock

Conversation

@JukkaL

@JukkaL JukkaL commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

The new native Lock class is a partial replacement for threading.Lock. It's much faster than threading.Lock in compiled code. In a microbenchmark I saw ~2.5x to ~4x performance improvement over compiled code using stdlib.

The Lock type has four platform-specific backends:

  • PyMutex on Python 3.14+ (all platforms); uses CPython's internal 1-byte atomic lock
  • SRWLOCK + condition variable on Windows < 3.14
  • POSIX semaphore on Linux < 3.14 with GIL
  • pthread mutex + condvar fallback (macOS, free-threaded builds)

All backends preserve threading.Lock semantics (cross-thread release allowed) and drop the GIL while blocking.

Added optimized handling of with lock: statements (for librt.threading only).

There is one significant known regression compared to CPython (beyond missing features): if two threads race to release the same lock, this triggers undefined behavior. The PyMutex C API doesn't provide a way to avoid this without a massive performance cost, so I decided to keep this limitation instead of sacrificing performance, as performance is the main goal of the new type.

This is just a lock -- using it with threading.Condition is not supported. We may later decide to add a native Condition (and possibly other synchronization primitives).

I used heavy coding agent assist with small, incremental changes, and multiple review iterations. I left the rather verbose comments generated by coding agents, since the code is legitimately tricky.

@github-actions

This comment has been minimized.

Comment on lines +229 to +236
// Fast path: grab the lock without releasing the GIL if it is free.
AcquireSRWLockExclusive(&self->srw);
if (!self->locked) {
self->locked = 1;
ReleaseSRWLockExclusive(&self->srw);
return 1;
}
ReleaseSRWLockExclusive(&self->srw);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is the same as the non-blocking path so we could drop the non-blocking path. instead always try the fast path and return 0 on failure if !blocking.

Comment on lines +271 to +280
{
int status;
do {
status = sem_trywait(&self->sem);
} while (status == -1 && errno == EINTR);
if (status == 0) {
self->locked = 1;
return 1;
}
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here - could merge the non-blocking/fast paths.

Comment on lines +326 to +333
// Fast path: grab the lock without releasing the GIL if it is free.
pthread_mutex_lock(&self->mut);
if (!self->locked) {
self->locked = 1;
pthread_mutex_unlock(&self->mut);
return 1;
}
pthread_mutex_unlock(&self->mut);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here.

Comment thread mypyc/lib-rt/threading/librt_threading.c Outdated
Comment thread mypyc/lib-rt/threading/librt_threading.c Outdated
JukkaL and others added 3 commits July 7, 2026 17:34
Co-authored-by: Piotr Sawicki <sawickipiotr@outlook.com>
Co-authored-by: Piotr Sawicki <sawickipiotr@outlook.com>
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

@JukkaL JukkaL merged commit f3294a0 into master Jul 7, 2026
25 checks passed
@JukkaL JukkaL deleted the librt-threading-lock branch July 7, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants