Skip to content

Update loops in CpuMath to be more efficient - #1177

Merged
shauheen merged 4 commits into
dotnet:masterfrom
jwood803:cpu-loops
Oct 31, 2018
Merged

shauheen merged 4 commits into
dotnet:masterfrom
jwood803:cpu-loops

Conversation

@jwood803

@jwood803 jwood803 commented Oct 6, 2018

Copy link
Copy Markdown
Contributor

Fixes issue #835

@Ivanidzo4ka

Copy link
Copy Markdown
Contributor

Did you run any benchmarks to make sure it bring speed improvements?

@sfilipi
sfilipi requested a review from tannergooding October 8, 2018 16:30
@danmoseley

Copy link
Copy Markdown

@jwood803 the end to end benchmarks are documented here. They take a bit of time to run.

There are micro benchmarks for CPUMath specifically which might be best to start with. They are here

@danmoseley

danmoseley commented Oct 11, 2018 •

Copy link
Copy Markdown

Oh, I just realized this is a duplicate of #994. That was not linked properly to #835.

As you see in #994, it is unfortunately not at all clear that this will actually help. See comment.

@tannergooding perhaps we should close #835 without action.

Vector128<float> scalarVector128 = Sse.SetAllVector128(scalar);

if (pDstCurrent + 4 <= pDstEnd)
if (pDstCurrent <= pDstEnd - 4)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

creating a temp for pDstEnd - 4 is probably better, as that would better ensure that each loop isn't doing pDstEnd - 4 before the comparison

@tannergooding

tannergooding commented Oct 11, 2018 •

Copy link
Copy Markdown
Member

@tannergooding perhaps we should close #835 without action.

I don't believe so, both PRs have the same problem, which is that each iteration of the loop may do a substraction/addition as part of the comparison.

As commented on #1177 (comment), creating a temp var pLoopEnd = pDstEnd - 4; and then using it in the comparison (while (pDstCurrent <= pLoopEnd)) will help ensure that the subtraction happens once, rather than once per loop iteration.

@jwood803

Copy link
Copy Markdown
Contributor Author

@danmosemsft Thanks a lot for letting me know about the benchmarks! I ran it two different times with the PredictionEngine benchmark. Below are the results. So far it looks like the changes provide a bit better performance.

Without changes:
image

With changes:
image

Vector128<float> scalarVector = Sse.SetAllVector128(scalar);

while (pDstCurrent + 4 <= pDstEnd)
while (pDstCurrent <= pDstEnd - 4)

@tannergooding tannergooding Oct 24, 2018 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you update the SSE case as well?


private const int Vector256Alignment = 32;

private const int destinationEnd = pDstEnd - 4;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is pDstEnd defined, I wouldn't expect this could be a constant....

@danmoseley

Copy link
Copy Markdown

@tannergooding does this look OK now?

Nice perf improvement @jwood803 thank you!

float* pDstEnd = pdst + dst.Length;
float* pDstCurrent = pdst;
int destinationEnd = pDstEnd - 4;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: stray newline here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be updated to remove the extra new line.

@tannergooding tannergooding left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shauheen
shauheen merged commit 71c9ff3 into dotnet:master Oct 31, 2018
@jwood803
jwood803 deleted the cpu-loops branch October 31, 2018 23:56
@eerhardt

eerhardt commented Nov 1, 2018

Copy link
Copy Markdown
Member

This was a bad change/merge

Error	CS0266	Cannot implicitly convert type 'float*' to 'int'. An explicit conversion exists (are you missing a cast?)	Microsoft.ML.CpuMath(netcoreapp3.0)	F:\git\machinelearning2\src\Microsoft.ML.CpuMath\SseIntrinsics.cs	758	Active
                float* pDstEnd = pdst + dst.Length;
                float* pDstCurrent = pdst;
                int destinationEnd = pDstEnd - 4;

pDstEnd is a float*. Subtracting 4 still gives you a float*, which is not convertable to int.

The Intrinsics/netcoreapp3.0 build is broken because of this.

@shauheen @danmosemsft - maybe now would be a good time to get a netcoreapp3.0 CI leg running.

I've opened #1495 for the build break.

@ghost ghost locked as resolved and limited conversation to collaborators Mar 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants