You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
The hashed values stored in the annotations indices are skewed by -1 #3072
// Output Data
//
// The original value of the 6012 category is MLS
// The original value of the 19014 category is NFL
// The original value of the 36205 category is MLB
Working as intended I believe, though perhaps some of the infrastructure surrounding this could be improved. Remember: we have key-values presented as indices into an enumerated set (so 0 must be the first valid value), but we have simultaneously declared that the default value for keys (which must be 0), should represent the missing value, rather than any particular element.
So 0 is and must be the "missing" key. The alternative is that it become some arbitrary member of whatever set is being enumerated, which is an utterly ridiculous proposition. We are also constrained by the fact that we are operating in an environment with 0 indexing. These two constraints force this compromise, which is, despite misunderstandings like this, the correct compromise.
Perhaps we could repurpose this example and documentation so as to properly explain to people what keys are.
@TomFinley adding a bit more to this, as i think this appearance of skewed keys and mapping 0 to the missing value might be harder to solve through documentation. As i was working on the samples for the KeyToValue, KeyToVector seeing the values loaded being skewed is confusing.. Would it be better to change the mapping of the missing value to be something like MaxInt? cc @Ivanidzo4ka
Closing after discussing alternatives like mapping the missing value to maxInt, or switching the rawType of KeyTypes to int?.
Mapping to something other than the default value is not possible, since uninitialized variables will initialize to that.
Trying to use Nullable uints showed a perf hit, during testing.
Activity
TomFinley commentedon Mar 22, 2019
Working as intended I believe, though perhaps some of the infrastructure surrounding this could be improved. Remember: we have key-values presented as indices into an enumerated set (so 0 must be the first valid value), but we have simultaneously declared that the default value for keys (which must be 0), should represent the missing value, rather than any particular element.
So 0 is and must be the "missing" key. The alternative is that it become some arbitrary member of whatever set is being enumerated, which is an utterly ridiculous proposition. We are also constrained by the fact that we are operating in an environment with 0 indexing. These two constraints force this compromise, which is, despite misunderstandings like this, the correct compromise.
Perhaps we could repurpose this example and documentation so as to properly explain to people what keys are.
TomFinley commentedon Mar 22, 2019
So... remove bug, maybe rephrase as documentation? The sample must at least be improved.
sfilipi commentedon Apr 4, 2019
@TomFinley adding a bit more to this, as i think this appearance of skewed keys and mapping 0 to the missing value might be harder to solve through documentation. As i was working on the samples for the KeyToValue, KeyToVector seeing the values loaded being skewed is confusing.. Would it be better to change the mapping of the missing value to be something like MaxInt? cc @Ivanidzo4ka
sfilipi commentedon Apr 6, 2019
Closing after discussing alternatives like mapping the missing value to maxInt, or switching the rawType of KeyTypes to int?.
Mapping to something other than the default value is not possible, since uninitialized variables will initialize to that.
Trying to use Nullable uints showed a perf hit, during testing.
For more on KeyTypes see: https://github.com/dotnet/machinelearning/blob/master/docs/code/IDataViewTypeSystem.md#key-types
https://github.com/dotnet/machinelearning/blob/master/docs/code/KeyValues.md