Skip to content

Introduce VBuffer GetValues and GetIndices - #1536

Merged
eerhardt merged 4 commits into
dotnet:masterfrom
eerhardt:VBufferGetValues
Nov 8, 2018
Merged

eerhardt merged 4 commits into
dotnet:masterfrom
eerhardt:VBufferGetValues

Conversation

@eerhardt

@eerhardt eerhardt commented Nov 6, 2018 •

Copy link
Copy Markdown
Member

Use the new methods in place of Count and Values/Indices in as many readonly situations as possible.

Working towards #608

See proposed change (4) in #608 (comment)

Eventually, Count, Values and Indices will be hidden/private on VBuffer and GetValues() and GetIndices() will be the only way to read these values.

@TomFinley TomFinley left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems quite positive @eerhardt, thank you. Fewer files than I expected too, somehow, but I guess these are the read-only applications. (All of them? That's fine if so, but I expected more somehow. Of course if you're just trying to stage the work in reasonable chunks that's totally fine too.)

Comment thread src/Microsoft.ML.Core/Data/VBuffer.cs Resolved
{
Contracts.AssertValueOrNull(input);
Contracts.Assert(0 <= min & min <= lim & lim <= Utils.Size(input));
return FindIndexSorted(input.AsSpan(), min, lim, value);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

input.AsSpan() [](start = 35, length = 14)

How heavy is AsSpan operation?
I would rather had two copies of same code for Array and ReadOnlySpan than perf degradation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How heavy is AsSpan operation?

It is super lightweight. There is actually in implicit conversion between array and span. It is only necessary here to tell the compiler to call the Span overload instead of the array overload.

Comment thread src/Microsoft.ML.FastTree/FastTree.cs Outdated
for (int i = 0; i < values.Count; ++i)
var valuesValues = values.GetValues();
var valuesIndices = values.GetIndices();
List<KeyValuePair<int, int>> result = new List<KeyValuePair<int, int>>();

@Ivanidzo4ka Ivanidzo4ka Nov 7, 2018 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List<KeyValuePair<int, int>> [](start = 12, length = 28)

nit: var?
it just stand out:)
also any reason why you prefer return IEnumerable to yield?

@eerhardt eerhardt Nov 7, 2018 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll fix it the var here.

also any reason why you prefer return IEnumerable to yield?

It is required because of the Span usage. You can't use a Span in a yield method because you can't hold onto a Span across the yield invocations.

@Ivanidzo4ka

Copy link
Copy Markdown
Contributor
    private static IEnumerable<KeyValuePair<int, int>> NonZeroBinnedValuesForSparse(VBuffer<double> values, Double[] binUpperBounds)

this thing got called once. so maybe it make sense to pass values.GetValues and values.GetIndicies here and call them values and indicies? It just weird to have valuesValues names in code:)


Refers to: src/Microsoft.ML.FastTree/FastTree.cs:1045 in 2de8d1f. [](commit_id = 2de8d1f, deletion_comment = False)

@Ivanidzo4ka

Copy link
Copy Markdown
Contributor
    private static IEnumerable<KeyValuePair<int, int>> NonZeroBinnedValuesForSparse(VBuffer<double> values, Double[] binUpperBounds)

On other thing you have this valuesValues everythere :) so feel free to ignore ^ comment.


In reply to: 436738591 [](ancestors = 436738591)


Refers to: src/Microsoft.ML.FastTree/FastTree.cs:1045 in 2de8d1f. [](commit_id = 2de8d1f, deletion_comment = False)

Comment thread src/Microsoft.ML.PCA/PcaTrainer.cs Resolved

@Ivanidzo4ka Ivanidzo4ka left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@Ivanidzo4ka

Copy link
Copy Markdown
Contributor

Out of curiosity, how is benchmarks looks like?

@eerhardt

eerhardt commented Nov 7, 2018

Copy link
Copy Markdown
Member Author

Out of curiosity, how is benchmarks looks like?

Practically identical, and within the margin of error:

Without Changes:

           Method |     Mean |    Error |   StdDev | Extra Metric |
----------------- |---------:|---------:|---------:|-------------:|
 TrainKMeansAndLR | 240.5 ms | 15.01 ms | 17.28 ms |            - |
 TrainKMeansAndLR | 242.0 ms | 10.75 ms | 11.50 ms |            - |
 TrainKMeansAndLR | 257.4 ms | 25.66 ms | 29.55 ms |            - |

With Changes:

           Method |     Mean |    Error |   StdDev | Extra Metric |
----------------- |---------:|---------:|---------:|-------------:|
 TrainKMeansAndLR | 244.2 ms | 8.233 ms | 8.086 ms |            - |
 TrainKMeansAndLR | 264.8 ms | 24.91 ms | 28.68 ms |            - |
 TrainKMeansAndLR | 249.1 ms | 14.35 ms | 16.52 ms |            - |

Use the new methods in place of Count and Values/Indices in as many readonly situations as possible.

Working towards dotnet#608
Update the rest of the BinaryWriter utils to use Span.
@eerhardt
eerhardt merged commit 18f7acc into dotnet:master Nov 8, 2018
@eerhardt
eerhardt deleted the VBufferGetValues branch November 8, 2018 05:24
@ghost ghost locked as resolved and limited conversation to collaborators Mar 27, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants