Repository navigation
Conversation
|
The following commit authors need to sign the Contributor License Agreement: |
Documentation build overview
56 files changed ·
|
|
@daanx can you please sign the CLA? |
Ah yes, but not really for the current PR as that includes the full mimalloc v3 vendored sources and that is a separate project under the MIT license. I can of course sign off for the changes I made on the CPython side. The current PR contains both, maybe we can split it? I think commit cedd47b is the CPython part only, while the other commits are just importing mimalloc v3 as a vendored library. |
daanx
left a comment
There was a problem hiding this comment.
Using MI_WIN_INIT_USE_FLS is not really encouraged for mimalloc. If we can use the default that would be best.
There's a crash on the Windows builds w/o it currently (and it seems to show up basically at startup but the freeze programs make it through it). I was planning on digging into it more later but this is what Claude came up as a fix - I just don't have immediate access to a Windows machine to see where the actual crash is coming from. But that's why it's there for now. Right now I'm just trying to get things to green CI and then I'll clean up the final result. |
|
@corona10 Hey here it is! |
|
About guardrail for memory pressure
|
|
Perf run is here waiting for results: https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/39346123598 |
FWIW the docs due already call out mimalloc license: https://docs.python.org/3/license.html#mimalloc although we should probably update the date range w/ the v3 update. |
2x to 1.14x but still higher than mimalloc v2 |
| } | ||
| } | ||
| // Hooks can run under allocator locks, so only consult cached progress. |
There was a problem hiding this comment.
| // Hooks can run under allocator locks, so only consult cached progress. | |
| if (qsbr != NULL) { | |
| // Scan thread sequences like the v2 integration did. | |
| return !_Py_qsbr_poll(qsbr, page->goal); | |
| } | |
| // Hooks can run under allocator locks, so only consult cached progress. |
That was still a 6% regression on memory (but things are headed in the right direction, THP tweak about it from 2x to 1.15, and then the _Py_qsbr_poll fix got us here), here's another run https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/39476892845 which has commit 3c8be4f that sets |
|
Another run with new options for mi_option_arena_purge_min_size, and mi_option_page_full_retain and a tweaked mi_option_arena_purge_mult: https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/39497633255 |
|
Down to 3% |
|
This is now neutral: https://github.com/facebookexperimental/free-threading-benchmarking/tree/main/results/bm-20261020-3.16.0a1+dev-a36efb6-NOGIL It includes one more upstream fix which is a36efb6 that re-enables the @daanx Thanks for merging all of the other upstream PRs I sent. I can send another one to re-enable this but I can also throw it under another ifdef if you're concerned about having it on in the common case. But I think this basically brings us to parity where we could merge this. I'd still want to bring back the latest commits you have upstream but my previous run had that as neutral. I also am assuming your dev3 branch isn't yet released? |
|
Another benchmark run w/ the latest and greatest mimalloc/dev3 branch merged in: https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/39651451425 We're now down to 4 vendored in changes which I've opened upstream PRs for (the last being the most questionable I think): |
Vendor mimalloc v3 (dev3 branch) into Objects/mimalloc and Include/internal/mimalloc.
|
One more run after merging against and updating the define for 3.6 #160360 https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/41690003577 |
|
It looks like the slice shift tweak hits assertions, doing another perf run w/o it and with the base having it: https://github.com/facebookexperimental/free-threading-benchmarking/actions/runs/41693317724 |
|
While looking into #161420, I tried this PR. In #161420, worker threads allocate 16 MiB The v3 code in this PR would fix that, but one of the option defaults the PR sets for CPython turns the fix off. Why v3 fixes it. With its default options, v3 abandons a page that becomes full: the page no longer belongs to the allocating thread ( Why it comes back here. This PR sets Peak memory, with about 150 MiB needed:
Could abandoning stay enabled, at least for pages of large blocks? The C program and all numbers are in this comment on microsoft/mimalloc#1117. |
…e aggressively collected (see also python/cpython#159718)
|
Hi @maikschulze , @DinoV : I agree that using However, we should try again also with the default page_full_retain (2). I think there is a mistake in the current PR: op->ob_tid = mi_page_is_abandoned(page) ? 0 : mi_page_thread_id(page);
if (op->ob_tid == 0) merge_refcount(op, 0);this should become: op->ob_tid = mi_thread_id_of(op); // this gets a CPython thread id (not a mimalloc abandoned page tid) (todo: make this more efficient by avoiding overwriting the tid in the object header (and avoiding this call)?
if (op->ob_tid == 0) merge_refcount(op, 0);edit: actually, it would be even better to call the internal I think with this change we don't need the page_full_retain and can use abandoned pages (which should improve matters). Fingers crossed :-) (not sure if you can easily pull from the latest |
|
Thank you, @daanx! I tried the C version of the example (8 threads, 16 MiB blocks freed by the main thread, about 144 MiB in flight) on the latest
|
|
Good see that @maikschulze; @DinoV , I tested the latest mimalloc dev3 with the fix I proposed: --- a/Python/gc_free_threading.c
+++ b/Python/gc_free_threading.c
@@ -311,8 +311,8 @@ gc_restore_tid(PyObject *op)
op->ob_tid = 0;
}
else {
- // Reclaimed pages may have a new owner; abandoned pages have none.
- op->ob_tid = mi_page_is_abandoned(page) ? 0 : mi_page_thread_id(page);
+ // Full-page abandonment preserves the QSBR thread ownership.
+ op->ob_tid = mi_page_qsbr_thread_id(page);
if (op->ob_tid == 0) {
merge_refcount(op, 0);
}and it all seems to work now with default mimalloc settings, i.e. I used the following (agent generated) test script that tries to probe threads creating abandoned pages that get picked up by other threads and gc'd:import argparse
from collections import Counter
from concurrent.futures import ThreadPoolExecutor
import ctypes
import faulthandler
import gc
import threading
import sys
import weakref
parser = argparse.ArgumentParser()
parser.add_argument("--reclaim", action="store_true")
args = parser.parse_args()
if sys.implementation.name != "cpython" or sys._is_gil_enabled():
raise SystemExit("Requires free-threaded CPython with the GIL disabled")
TIMEOUT = 30
COUNT = 20_000
faulthandler.dump_traceback_later(TIMEOUT, exit=True)
gc.disable()
def object_thread_id(obj):
# ob_tid is the first word of a free-threaded CPython object.
return ctypes.c_size_t.from_address(id(obj)).value
class Node:
__slots__ = ("value", "link", "__weakref__")
def __init__(self, value):
self.value = value
self.link = None
state = {}
ready = threading.Event()
reclaimed = threading.Event()
finish = threading.Event()
def allocate():
objects = [Node(i) for i in range(COUNT)]
state["owner"] = object_thread_id(objects[0])
state["freed"] = {id(objects[i]) for i in range(0, COUNT, 2)}
state["garbage"] = [
weakref.ref(objects[i]) for i in range(0, COUNT, 2)
]
# Half become unreachable cycles, to be freed by GC on another thread.
for i in range(0, COUNT, 2):
objects[i].link = objects[i]
objects[i] = None
state["objects"] = objects
ready.set()
assert finish.wait(TIMEOUT)
return sum(obj.value for obj in objects if obj is not None)
def reclaim_pages():
objects = [Node(i) for i in range(COUNT)]
state["reclaimer"] = object_thread_id(objects[0])
state["reused"] = sum(
id(obj) in state["freed"] for obj in objects
)
state["new"] = objects
reclaimed.set()
assert finish.wait(TIMEOUT)
return sum(
obj.value for obj in state["objects"] if obj is not None
)
expected_sum = sum(range(1, COUNT, 2))
with ThreadPoolExecutor(max_workers=2) as pool:
owner = pool.submit(allocate)
other = None
try:
assert ready.wait(TIMEOUT)
objects = state["objects"]
refs = [
weakref.ref(obj) for obj in objects if obj is not None
]
assert {
object_thread_id(obj)
for obj in objects if obj is not None
} == {state["owner"]}
# Free cycles outside their allocating thread, avoiding owner reclaim.
gc.collect()
assert all(ref() is None for ref in state["garbage"])
expected_owners = {state["owner"]}
if args.reclaim:
other = pool.submit(reclaim_pages)
assert reclaimed.wait(TIMEOUT)
assert state["reused"] > 0, "No freed slots were reused"
expected_owners.add(state["reclaimer"])
for _ in range(2):
gc.collect()
owners = Counter(
object_thread_id(obj)
for obj in objects if obj is not None
)
assert owners.keys() <= expected_owners, owners
if args.reclaim:
assert owners[state["reclaimer"]] > 0, owners
assert sum(
obj.value for obj in objects if obj is not None
) == expected_sum
finally:
finish.set()
assert owner.result() == expected_sum
if other is not None:
assert other.result() == expected_sum
del objects
state.clear()
gc.collect()
assert all(ref() is None for ref in refs)
faulthandler.cancel_dump_traceback_later()
print("PASS:", "reclaimed pages" if args.reclaim else "abandoned pages")To run both scenarios with your free-threaded interpreter: for retain in 0 2; do
MIMALLOC_PAGE_FULL_RETAIN=$retain MIMALLOC_PAGE_RECLAIM_ON_FREE=0 \
./python -X gil=0 qsbr_gc_test.py
MIMALLOC_PAGE_FULL_RETAIN=$retain MIMALLOC_PAGE_RECLAIM_ON_FREE=0 \
./python -X gil=0 qsbr_gc_test.py --reclaim
done |
|
Thanks, I reran the #161420 pattern against this PR, in both directions, in case it helps with the On speed: 2.5.2 makes an
On memory, the pattern: 16 MiB
Reproducer (standard library only)# Large blocks made on one thread and freed on another, 8 in flight: 10 rounds of 64 blocks of 16 MiB.
# usage: python pool_repro.py produce|consume new|keep [gc]
# produce: workers make the blocks, the main thread frees them (as in gh-161420)
# consume: the main thread reads the blocks from a file, workers checksum and free them
# new: a new ThreadPoolExecutor per round (the workers exit); keep: one pool for all rounds
# gc: call gc.collect() after every round
import gc, resource, sys, zlib
from collections import deque
from concurrent.futures import ThreadPoolExecutor
direction, mode, collect = sys.argv[1], sys.argv[2], sys.argv[3:] == ["gc"]
BLOCK, COUNT, THREADS, ROUNDS = 16 << 20, 64, 8, 10
pattern = bytes(range(256)) * (BLOCK // 256)
source = open("/dev/zero", "rb", buffering=0)
def make():
return memoryview(pattern).tobytes() # a new bytes object, as decompress() returns
kept = ThreadPoolExecutor(THREADS) if mode == "keep" else None
for _ in range(ROUNDS):
pool = kept or ThreadPoolExecutor(THREADS)
pending = deque()
for _ in range(COUNT):
if direction == "produce":
pending.append(pool.submit(make)) # the block is freed on the main thread
else:
pending.append(pool.submit(zlib.crc32, source.read(BLOCK))) # freed on the worker
if len(pending) >= THREADS:
pending.popleft().result()
while pending:
pending.popleft().result()
if kept is None:
pool.shutdown() # the workers exit
if collect:
gc.collect()
peak = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
peak = peak >> 20 if sys.platform == "darwin" else peak >> 10
print(f"{direction} {mode}{' gc' if collect else ''}: peak RSS {peak} MiB")for retain in -1 2; do
for args in "produce keep" "consume keep" "produce new" "produce new gc" "consume keep gc"; do
MIMALLOC_PAGE_FULL_RETAIN=$retain ./python pool_repro.py $args
done
doneA probe that suggests the waiting pages are the whole difference (for tracing only, since it lets memory be reused while lock-free readers may still look at it): returning false at the top of if (!mi_heap_qsbr_is_enabled(heap)) return false;
+ return false; // PROBE ONLY: never delay empty pages for QSBR (unsafe in general)
const bool delayed = mi_page_qsbr_should_delay(&page->qsbr, &heap->qsbr, page->capacity * mi_page_block_size(page));The counters are a small patch I can share if useful. Separately: in v3, a block above 31 MiB seems to be purged on every free. As far as I can see, I used an AI assistant to run these measurements and draft this comment; I reviewed the results myself. |
|
Thanks @maikschulze -- I pushed an update to the latest |
|
Thanks, that was quick! With
You're right that the benchmark is synthetic, so, with an AI assistant's help, I put the same builds under a real program: a package manager that extracts
(2.5.2 is main, v3 is 80899607d2; GIL builds with
As an experiment only, I let |
|
Thanks @maikschulze ; The latter issue you raised turned out to be more tricky with the increased RSS. I think I found a reasonable solution though where abandoned empty qsbr-delayed pages from threads that have exited are now retained in a heap qsbr_delayed list. I think this will fix the issue -- can you try it ? It is in the latest |
|
Thanks, that does it here, also with v3.7.0 (released 2026-11-17). In the package manager where this started, the peak is back at the level of 2.5.2 (631 against 601 MiB on macOS, with 9548089b04), and
The v3 builds are this PR merged into main with the mimalloc version in question; Linux arm64 (Debian 13, GCC 14.2), free-threaded release builds unless noted; the build steps are in the last fold. It also got faster where workers exit. Blocks made by 8 workers and freed by the main thread, a new pool per round, GiB/s (release builds, fastest of 5 rounds of 1 GiB, median of 5 processes; measured before v3.7.0):
Where each worker frees its own block, the two Beyond that, I compared v3.7.0 with main (2.5.2) more widely: the full test suite on a release build, 1. A forked child can hangI noticed this through So it takes The fork notes give CPython's stop-the-world as an example of all other threads being at a safe point outside mimalloc. As far as I can tell it doesn't cover these starting and ending threads, so this may need something on the CPython side of this PR, and perhaps a word in the notes. Letting The loop, and the sketch I used
import os, sys, threading, time
def do_fork(pids):
pid = os.fork()
if pid == 0:
if sys.argv[2:] == ["thread"]:
t = threading.Thread(target=lambda: None)
t.start()
t.join()
os._exit(50)
pids.append(pid)
for r in range(int(sys.argv[1])):
pids = []
threads = [threading.Thread(target=do_fork, args=(pids,)) for _ in range(16)]
for t in threads:
t.start()
for t in threads:
t.join()
deadline = time.monotonic() + 20
for pid in pids:
while os.waitpid(pid, os.WNOHANG)[0] == 0:
if time.monotonic() > deadline:
sys.exit(f"child {pid} of round {r} is still running")
time.sleep(0.01)
print("no hang")The sketch, against this PR at 48883a2 merged into main at e8c0584: --- a/Include/internal/pycore_pystate.h
+++ b/Include/internal/pycore_pystate.h
@@ -242,6 +242,7 @@
PyInterpreterState *interp,
int whence);
extern void _PyThreadState_Bind(PyThreadState *tstate);
+extern int _PyThreadState_BindUnlessMustExit(PyThreadState *tstate);
PyAPI_FUNC(PyThreadState *) _PyThreadState_NewBound(
PyInterpreterState *interp,
int whence);
@@ -249,6 +250,16 @@
extern void _PyThreadState_DeleteList(PyThreadState *list, int is_after_fork);
extern void _PyThreadState_ClearMimallocHeaps(PyThreadState *tstate);
+#ifdef Py_GIL_DISABLED
+// Held by a thread while it runs code before it is attached or after its
+// thread state is gone that takes locks a stop-the-world cannot see
+// (mimalloc, the table of biased reference counting), and by os.fork()
+// across the fork, so that the child does not inherit them half done.
+extern void _PyThreadState_UnattachedLock(void);
+extern void _PyThreadState_UnattachedUnlock(void);
+extern void _PyThreadState_UnattachedReInit(void);
+#endif
+
// Export for '_testinternalcapi' shared extension
PyAPI_FUNC(PyObject*) _PyThreadState_GetDict(PyThreadState *tstate);
--- a/Modules/_threadmodule.c
+++ b/Modules/_threadmodule.c
@@ -9,6 +9,7 @@
#include "pycore_moduleobject.h" // _PyModule_GetState()
#include "pycore_object_deferred.h" // _PyObject_SetDeferredRefcount()
#include "pycore_pylifecycle.h"
+#include "pycore_mimalloc.h" // mi_thread_done()
#include "pycore_pystate.h" // _PyThreadState_SetCurrent()
#include "pycore_time.h" // _PyTime_FromSeconds()
#include "pycore_tuple.h" // _PyTuple_FromPairSteal
@@ -371,7 +372,7 @@
// _PyRuntimeState_SetFinalizing() is called. At this point, all Python
// threads must exit, except of the thread calling Py_Finalize() which
// holds the GIL and must not exit.
- if (_PyThreadState_MustExit(tstate)) {
+ if (!_PyThreadState_BindUnlessMustExit(tstate)) {
// Don't call PyThreadState_Clear() nor _PyThreadState_DeleteCurrent().
// These functions are called on tstate indirectly by Py_Finalize()
// which calls _PyInterpreterState_Clear().
@@ -382,7 +383,6 @@
goto exit;
}
- _PyThreadState_Bind(tstate);
PyEval_AcquireThread(tstate);
_Py_atomic_add_ssize(&tstate->interp->threads.count, 1);
@@ -406,6 +406,15 @@
PyThreadState_Clear(tstate);
_PyThreadState_DeleteCurrent(tstate);
+#if defined(Py_GIL_DISABLED) && defined(WITH_MIMALLOC)
+ // Let mimalloc give up this thread's heaps here, under the lock that
+ // os.fork() holds across the fork, and not only in its thread-exit
+ // handler, which runs when no stop-the-world can wait for this thread.
+ _PyThreadState_UnattachedLock();
+ mi_thread_done();
+ _PyThreadState_UnattachedUnlock();
+#endif
+
exit:
// Don't need to wait for this thread anymore
remove_from_shutdown_handles(handle);
--- a/Modules/posixmodule.c
+++ b/Modules/posixmodule.c
@@ -715,6 +715,9 @@
_PyImport_AcquireLock(interp);
_PyEval_StopTheWorldAll(&_PyRuntime);
+#ifdef Py_GIL_DISABLED
+ _PyThreadState_UnattachedLock();
+#endif
HEAD_LOCK(&_PyRuntime);
_PyWarnings_BeforeFork(interp);
}
@@ -723,6 +726,9 @@
PyOS_AfterFork_Parent(void)
{
HEAD_UNLOCK(&_PyRuntime);
+#ifdef Py_GIL_DISABLED
+ _PyThreadState_UnattachedUnlock();
+#endif
_PyEval_StartTheWorldAll(&_PyRuntime);
PyInterpreterState *interp = _PyInterpreterState_GET();
@@ -753,6 +759,9 @@
PyStatus status;
_PyRuntimeState *runtime = &_PyRuntime;
+#ifdef Py_GIL_DISABLED
+ _PyThreadState_UnattachedReInit();
+#endif
#ifdef WITH_MIMALLOC
_mi_fork_child();
#endif
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -163,9 +163,40 @@
static void tstate_mimalloc_bind(PyThreadState *);
+#ifdef Py_GIL_DISABLED
+static PyMutex unattached_mutex;
+
+void
+_PyThreadState_UnattachedLock(void)
+{
+ PyMutex_LockFlags(&unattached_mutex, _Py_LOCK_DONT_DETACH);
+}
+
+void
+_PyThreadState_UnattachedUnlock(void)
+{
+ PyMutex_Unlock(&unattached_mutex);
+}
+
+void
+_PyThreadState_UnattachedReInit(void)
+{
+ unattached_mutex = (PyMutex){0};
+}
+#endif
+
+static void
+bind_tstate_impl(PyThreadState *tstate, int lock_held);
+
static void
bind_tstate(PyThreadState *tstate)
{
+ bind_tstate_impl(tstate, 0);
+}
+
+static void
+bind_tstate_impl(PyThreadState *tstate, int lock_held)
+{
assert(tstate != NULL);
assert(tstate_is_alive(tstate) && !tstate->_status.bound);
assert(!tstate->_status.unbound); // just in case
@@ -184,6 +215,12 @@
#endif
#ifdef Py_GIL_DISABLED
+ // This thread is not attached yet, so a stop-the-world does not wait for
+ // it; os.fork() holds this lock across the fork.
+ if (!lock_held) {
+ _PyThreadState_UnattachedLock();
+ }
+
// Initialize biased reference counting inter-thread queue. Note that this
// needs to be initialized from the active thread.
_Py_brc_init_thread(tstate);
@@ -192,6 +229,14 @@
// mimalloc state needs to be initialized from the active thread.
tstate_mimalloc_bind(tstate);
+#ifdef Py_GIL_DISABLED
+ if (!lock_held) {
+ _PyThreadState_UnattachedUnlock();
+ }
+#else
+ (void)lock_held;
+#endif
+
tstate->_status.bound = 1;
}
@@ -2826,6 +2871,30 @@
}
}
+// For a new thread: bind its thread state unless the interpreter is shutting
+// down. Returns 0 if the thread must exit. The check and the bind are done
+// under the lock that os.fork() holds across a fork, so that waiting for it
+// cannot come between the two.
+int
+_PyThreadState_BindUnlessMustExit(PyThreadState *tstate)
+{
+#ifdef Py_GIL_DISABLED
+ _PyThreadState_UnattachedLock();
+#endif
+ int must_exit = _PyThreadState_MustExit(tstate);
+ if (!must_exit) {
+ assert(_PyThreadState_CheckConsistency(tstate));
+ bind_tstate_impl(tstate, 1);
+ if (gilstate_get() == NULL) {
+ bind_gilstate_tstate(tstate);
+ }
+ }
+#ifdef Py_GIL_DISABLED
+ _PyThreadState_UnattachedUnlock();
+#endif
+ return !must_exit;
+}
+
#if defined(Py_GIL_DISABLED) && !defined(Py_LIMITED_API)
uintptr_t
_Py_GetThreadLocal_Addr(void)2. A debug-build assertionWith A thread exits and leaves a partly used page, another thread frees some of its objects, then a collection visits the page: import gc, threading
class C:
pass
keep = []
def worker():
keep.append([C() for _ in range(100)])
t = threading.Thread(target=worker); t.start(); t.join()
del keep[0][::2] # the main thread frees half of the worker's objects
gc.collect() # the assertion fires hereThe blocks freed by the other thread are still on 3. A warning at exit in debug buildsCPython deletes its heaps at interpreter exit without calling 4. The data race of #129748 is backFreeing a block writes the free-list link into the block's first word, which is the object's
The TSan job here builds with The change, and the script--- a/include/mimalloc/alloc/page.h
+++ b/include/mimalloc/alloc/page.h
@@ -816,7 +816,7 @@
static inline mi_encoded_t mi_block_link(const mi_block_t* block) {
mi_track_mem_defined(block,sizeof(mi_block_t));
- const mi_encoded_t x = block->next;
+ const mi_encoded_t x = mi_atomic_load_relaxed((_Atomic(mi_encoded_t)*)&block->next);
mi_track_mem_noaccess(block,sizeof(mi_block_t));
return x;
}
@@ -833,7 +833,7 @@
static inline void mi_block_set_next(const mi_page_t* page, mi_block_t* block, const mi_block_t* next) {
mi_track_mem_undefined(block,sizeof(mi_block_t));
- block->next = mi_block_encode(page, block, next);
+ mi_atomic_store_relaxed((_Atomic(mi_encoded_t)*)&block->next, mi_block_encode(page, block, next));
mi_track_mem_noaccess(block,sizeof(mi_block_t));
}
Four readers load a slot while one writer replaces its value: import sys, threading, time
class C:
__slots__ = ("x",)
seconds = float(sys.argv[1]) if len(sys.argv) > 1 else 5.0
c = C()
c.x = object()
stop = []
def reader():
while not stop:
for _ in range(1000):
c.x
def writer():
while not stop:
for _ in range(1000):
c.x = object() # frees the previous value
threads = [threading.Thread(target=reader) for _ in range(4)] + [threading.Thread(target=writer)]
for t in threads:
t.start()
time.sleep(seconds)
stop.append(1)
for t in threads:
t.join()5. Two memory properties
The scripts, and the probe
# Phases with small bytes objects of one size each. Per phase, 8 workers make 512 MiB of them, all
# kept; then the main thread frees them. At most 512 MiB (plus the lists) is alive.
# usage: python size_classes.py new|keep [gc]
# new: a new ThreadPoolExecutor per phase (the workers exit before the frees); keep: one pool for all phases
import gc, resource, sys
from concurrent.futures import ThreadPoolExecutor
mode, collect = sys.argv[1], sys.argv[2:] == ["gc"]
SIZES = [64, 100, 160, 250, 400, 640, 1000, 1600, 2500, 4000] # payload bytes: ten size classes
TOTAL, THREADS = 512 << 20, 8
def make(job):
size, n = job
return [b"x" * size for _ in range(n)]
kept = ThreadPoolExecutor(THREADS) if mode == "keep" else None
for size in SIZES:
pool = kept or ThreadPoolExecutor(THREADS)
parts = list(pool.map(make, [(size, TOTAL // (size + 33) // THREADS)] * THREADS))
if kept is None:
pool.shutdown() # the workers exit
while parts:
parts.pop() # freed on the main thread
if collect:
gc.collect()
peak = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
print("size_classes", *sys.argv[1:], "peak RSS", peak >> 20 if sys.platform == "darwin" else peak >> 10, "MiB")
# The workload of https://github.com/python/cpython/issues/145615 with the kind of object as a parameter, and a
# memory reading that also works on macOS (physical footprint there, resident set on Linux).
# python gh145615_kinds.py [ITERATIONS] [bytearray|bytes|list|dict|object] [nogc] [census]
# nogc: without the two gc.collect() calls per iteration; census: sys._debugmallocstats() at the end (diagnostic build).
# bytearray is the issue's own case (a small object plus a 64 byte buffer from PyMem_Malloc).
import concurrent.futures, gc, os, sys
if sys.platform == "darwin":
import ctypes
_buf = (ctypes.c_uint64 * 64)()
_lib = ctypes.CDLL(None)
def mem_mib():
_lib.proc_pid_rusage(os.getpid(), 4, ctypes.byref(_buf))
return _buf[2:][7] >> 20 # physical footprint
else:
def mem_mib():
with open("/proc/self/statm") as f:
return int(f.read().split()[1]) * os.sysconf("SC_PAGE_SIZE") >> 20
class C:
pass
MAKE = {
"bytearray": lambda: bytearray(64),
"bytes": lambda: bytes(64),
"list": lambda: [None] * 8,
"dict": lambda: {"a": 1},
"object": lambda: C(),
}
iterations = int(sys.argv[1]) if len(sys.argv) > 1 else 100
kind = sys.argv[2] if len(sys.argv) > 2 else "bytearray"
make = MAKE[kind]
use_gc = "nogc" not in sys.argv
def work(_):
return [make() for _ in range(4000)]
pool = concurrent.futures.ThreadPoolExecutor(max_workers=4)
readings = []
for i in range(iterations):
futures = [pool.submit(work, j) for j in range(100)]
results = [f.result() for f in futures]
del futures, results # free worker-allocated objects on main thread
if use_gc:
gc.collect()
gc.collect()
if i == 0 or (i + 1) % (iterations // 4) == 0:
readings.append(mem_mib())
if "census" in sys.argv:
sys._debugmallocstats()
pool.shutdown()
print(f"{kind}{'' if use_gc else ', no gc.collect()'}, {iterations} iterations: memory in MiB at iteration 1 and after each quarter: {', '.join(map(str, readings))}")The probe, on 9548089b04: --- a/src/alloc/alloc-generic.c
+++ b/src/alloc/alloc-generic.c
@@ -177,6 +177,10 @@
if (page == NULL) {
_mi_theap_collect_retired(theap, false); // perhaps make a page available
+ if (first_try && pq->last != NULL && mi_page_immediate_available(pq->last)) { // PROBE: use what the scan made available
+ page = mi_page_queue_find_free_ex(theap, pq, false);
+ if (page != NULL) { return page; }
+ }
page = _mi_page_fresh(theap, pq);
mi_assert_internal(page == NULL || mi_page_immediate_available(page));
if (page == NULL && first_try) {How the builds were madeThis PR merged into main, with a mimalloc release copied over the vendored files, as git checkout e8c0584877e0b0a82ee8e0707336d8df4d68ccb4
git merge 48883a281d72773b18c11b3ad0eddbb455f50f70 # conflicts in Objects/mimalloc/ only
curl -L https://github.com/microsoft/mimalloc/archive/refs/tags/v3.7.0.tar.gz | tar xz -C /tmp
find Objects/mimalloc -type f ! -name refresh.sh -delete
rm -rf Include/internal/mimalloc && mkdir Include/internal/mimalloc
cp -R /tmp/mimalloc-3.7.0/src/. Objects/mimalloc/
cp -R /tmp/mimalloc-3.7.0/include/. Include/internal/mimalloc/
cp /tmp/mimalloc-3.7.0/LICENSE Objects/mimalloc/
sed -i.bak 's/theap->qsbr_first != NULL/theap->qsbr_delayed != NULL/' Objects/obmalloc.c
./configure --disable-gil && make -j8 # --with-pydebug for items 2 and 3, --with-thread-sanitizer for item 4 |
This upgrades mimalloc to v3 - the initial work here was done by @daanx and this pushes it over the edge to getting all CI passing and conforming we're at parity with the existing implementation. v3 brings with it baked in qsbr support and it has a number of the CPython specific changes upstreamed now.
To maintain parity with v2 on the pyperformance benchmarks we set several options back to their v2 levels in
pycore_mimalloc.h.There's a new Github CI job added which took 15 seconds to run on this job that verifies we have now modified mimalloc. It is modeled after the
expatrefresh script / CI job. We sync to a specific mimalloc tag and have the hash recorded. It downloads and verifies that the mimalloc vendored code (which lives inObjects/mimallocandInclude/internal/mimalloc) is unchanged.