Skip to content

Lower the default THREAD_TIMEOUT from 28 to 22 - #6253

Open
ptweezy wants to merge 1 commit into
OpenMathLib:developfrom
ptweezy:thread-timeout-default
Open

ptweezy wants to merge 1 commit into
OpenMathLib:developfrom
ptweezy:thread-timeout-default

Conversation

@ptweezy

@ptweezy ptweezy commented Nov 14, 2026

Copy link
Copy Markdown

After a threaded call, each worker in blas_thread_server() busy-waits for 1 << THREAD_TIMEOUT timer cycles before it sleeps. The default of 28 is about 90 ms at 3 GHz. This changes it to 22, about 1.4 ms. -DTHREAD_TIMEOUT and OPENBLAS_THREAD_TIMEOUT still override it.

One isolated call costs (threads - 1) x 90 ms of CPU, and a program that calls more often than every 90 ms never lets the workers sleep. The CPU comes out of a second pool in the same process (the NumPy and SciPy wheels each bundle one), other processes, or a container's quota. #5171 and #1560 report this, and #1614 notes that the value is unchanged from GotoBLAS2.

Benchmarks

GitHub-hosted runners with 4 vCPUs, two VMs per architecture, default pool size. The Python rows use the NumPy 2.5.4 and SciPy 1.18.1 wheels with OPENBLAS_THREAD_TIMEOUT=22, which measured the same as the patched build. Scripts and full results are in https://github.com/ptweezy/openblas-thread-timeout-bench.

THREAD_TIMEOUT 22 vs. 28 x86-64 (EPYC 7763, 9V74) arm64 (Neoverse-N2)
25 NumPy/SciPy/scikit-learn workloads, wall time (geomean) 0.84x 0.88x
Same workloads, CPU time 0.50x 0.54x
Workloads more than 10% slower 0 0
2 worker processes, requests/s 146 to 236 133 to 278
Container limited to 2 of 4 CPUs, requests/s 92 to 161 91 to 179

The cost is waking the workers when a call arrives after they have slept: 15 to 20 µs per thread on a Ryzen 7 9800X3D. A 300x300 dgemm on 12 threads goes from 84 to 317 µs after a 10 ms gap, and back-to-back calls are unchanged. With 12 threads on that machine, no workload was more than 6% slower.

Shorter values regressed: 20 made one loop 1.7x slower with 8 threads, and 21 made one workload 1.3x slower with 12. I have no data above 16 CPUs or on bare metal. The wake-up costs more with a larger pool, so 24 would leave more margin there, but it gives up most of the gain in the last two rows.

After a threaded call, each worker in blas_thread_server() busy-waits
for 1 << THREAD_TIMEOUT timer cycles before it sleeps. 28 is about
90 ms at 3 GHz; 22 is about 1.4 ms. -DTHREAD_TIMEOUT and
OPENBLAS_THREAD_TIMEOUT still override it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant