Repository navigation
Poor dsyrk performance on skylake using more than 1 threads #1115
Description
I am getting poor performance for dsyrk when using more than 1 thread. My cpu is Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz, generation Skylake. Openblas defaults to using the haswell kernels.
I am looking at the perfomance of a simple X^T.dot(X) for X of size 1,000,000 by 100, using dsyrk and comparing it to dgemm for n=1 or 2 threads.
The performance numbers are summarized in the table below. They are time in s for 5 runs.
| test | Haswell | Sandybridge |
|---|---|---|
| dsyrk(2 threads) | 2.49 | 1.83 |
| dgemm(2 threads) | 2.09 | 2.86 |
| dsyrk (1 thread) | 2.60 | 3.13 |
| dgemm (1 thread) | 3.47 | 5.17 |
The numbers for target Sandybridge were run on the same machine, but by recompiling openblas and specifying TARGET=SANDYBRIDGE in the Makefile.
The second column numbers make sense: dsyrk is ~1.7 faster than dgemm, and we gain another 1.7 factor by doubling the number of threads. The haswell column is quite disappointing. dgemm seems to scale well, but dsyrk doesn't seem to scale at all with more than 1 thread, so that using dgemm with 2 threads is actually faster.
Has anybody observed something similar? I am wondering if this is related to #730, although I'm using linux. The script that I used to get those numbers is available here.
Activity
brada4 commentedon Mar 6, 2017
Which version of OpenBLAS?
Which operating system?
... Python ... Numpy ... CC ... ?
Same dsyrk is used for sandy bridge and haswell which is created from same C source file by same your compiler. Do you have any explanation to measured difference?
martin-frbg commentedon Mar 6, 2017
using (probably) differing cpu-specific constants from param.h to guide unrolling (b07d733 could be relevant) and using different assembly implementations of xGEMM to do the actual work...
thrasibule commentedon Mar 6, 2017
@brada4, this is with latest master on Linux. I've posted a link to the script I used to generate the numbers in the first post. I'm reposting it here I'm using numpy to call blas, but that shouldn't matter.
martin-frbg commentedon Mar 8, 2017
This may be a silly question, but have you checked if your test script is actually running on more than one core ?
thrasibule commentedon Mar 8, 2017
This is not a silly question at all, I think you're onto something! I have two cores with multithreading. If I do nothing, openblas will use 4 threads, but if I usually force it to 2, because the performance is slightly better.
But with latest master, dsyrk will only use 1 thread if I run it with OMP_NUM_THREADS=2, and 2 threads if OMP_NUM_THREADS=4. That explains the numbers that I reported, with dsyrk not being faster for 2 threads, I needs to use 4.
Now on a whim, I tried to revert b07d733 which you said might be relevant, and the behavior is more logical, with dsyrk using 2 threads if I set OMP_NUM_THREADS=2, and 4 if I set OMP_NUM_THREADS=4, which matches with how dgemm behaves. Performance is actually slightly better too, but that might be some measurement error.
Let me know if this makes sense. I don't understand what the rationale behind b07d733 is, so this is just poking in the dark for me. I can run more tests if that helps.
martin-frbg commentedon Mar 8, 2017
While I obviously cannot speak for wernsaar, it could be that the code was optimized for a different workload (your matrix size might be close to the number where he observed threading overhead to be greater than parallelization advantage). As an aside I am having problems trying to replicate your results on my i7-7500U at the moment - could be a hardware issue (or a borked numpy shipped with the distribution) as my timings from your script are on the order of 30 to 60 seconds.
martin-frbg commentedon Aug 1, 2017
Returning to this old issue, another factor may have been that - as was discovered in #1232 - until very recently the cpuid code misdetected the L1 cache on Haswell and later generation Intel chips as only a quarter of the actual size.
h-vetinari commentedon Nov 6, 2019
@thrasibule
Could you rerun the perf tests with latest master? There's been a bunch of changes to the skylake code, compare some of the merged PRs.
martin-frbg commentedon Nov 6, 2019
@h-vetinary the recent changes you mention are for "Skylake X" which is high-end Xeon with AVX512 - this ticket is about older i7-6xxx hardware
martin-frbg commentedon Nov 6, 2019
@h-vetinary the recent changes you mention are for "Skylake X" which is high-end Xeon with AVX512 - this ticket is about older i7-6xxx hardware
1 remaining item
martin-frbg commentedon Nov 6, 2019
Blame Intel for the confusing names - "Skylake X" or "Skylake Xeon" has AVX512 support which all the recent troubles and improvements in the "Skylake X" area was about, plain "Skylake" is just another slightly improved Haswell refresh in between "Broadwell" and "Kaby Lake". The issue here is "just" that a change appears to have been made than favors one kind of workload over another,
and the person who contributed this change stopped contributing (or even responding) at around the same time that this was brought up.
david-cortes commentedon Feb 15, 2020
Comment: the issue also affects AMD Ryzen 7 2700.
martin-frbg commentedon Oct 15, 2020
Pushing the milestone again but have confirmed that the introduction of the GEMM_UNROLL_MN for Haswell (with subsequent inheriting by newer generations) slows down both syrk and gemm at this problem size (at least). Best results when the parameter is removed (which gives a value of 8, compared to 16 before the dubious #747) and the additional condition for
>= 2 * GEMM_UNROLL_MNas well. I notice wernsaar apparently made that change in response to, but without commenting on, #730 which may have been affected by an OSX-specific issue (mishandled aligns in the assembly kernels) and additionally was about single-thread performance.martin-frbg commentedon Dec 11, 2020
fixed by reverting #747 in #3026