Repository navigation
Conversation
After a threaded call, each worker in blas_thread_server() busy-waits for 1 << THREAD_TIMEOUT timer cycles before it sleeps. 28 is about 90 ms at 3 GHz; 22 is about 1.4 ms. -DTHREAD_TIMEOUT and OPENBLAS_THREAD_TIMEOUT still override it.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After a threaded call, each worker in
blas_thread_server()busy-waits for1 << THREAD_TIMEOUTtimer cycles before it sleeps. The default of 28 is about 90 ms at 3 GHz. This changes it to 22, about 1.4 ms.-DTHREAD_TIMEOUTandOPENBLAS_THREAD_TIMEOUTstill override it.One isolated call costs (threads - 1) x 90 ms of CPU, and a program that calls more often than every 90 ms never lets the workers sleep. The CPU comes out of a second pool in the same process (the NumPy and SciPy wheels each bundle one), other processes, or a container's quota. #5171 and #1560 report this, and #1614 notes that the value is unchanged from GotoBLAS2.
Benchmarks
GitHub-hosted runners with 4 vCPUs, two VMs per architecture, default pool size. The Python rows use the NumPy 2.5.4 and SciPy 1.18.1 wheels with
OPENBLAS_THREAD_TIMEOUT=22, which measured the same as the patched build. Scripts and full results are in https://github.com/ptweezy/openblas-thread-timeout-bench.The cost is waking the workers when a call arrives after they have slept: 15 to 20 µs per thread on a Ryzen 7 9800X3D. A 300x300
dgemmon 12 threads goes from 84 to 317 µs after a 10 ms gap, and back-to-back calls are unchanged. With 12 threads on that machine, no workload was more than 6% slower.Shorter values regressed: 20 made one loop 1.7x slower with 8 threads, and 21 made one workload 1.3x slower with 12. I have no data above 16 CPUs or on bare metal. The wake-up costs more with a larger pool, so 24 would leave more margin there, but it gives up most of the gain in the last two rows.