Skip to content

LightGBM Error, code is -1, error message is 'bad allocation' #3340

Closed

Description

@PeterPann23

System information

  • **Windows 10 **:10.0.18362 Build 18362
  • .NET: Version 4.7.3, Microsoft.ML 1.0 preview
    -** Microsoft.ML.LightGMB**: 0.11.27505.13@build By dlab -14DDVS...

Issue

  • What did you do?
    Started training using following code:
var pipeline = mlContext.Transforms.Conversion.MapValueToKey(outputColumnName: nameof(PredictedResult.LabelIndex) , inputColumnName: nameof(Data.Label), sort: ValueToKeyMappingEstimator.SortOrder.Value)
                       .Append(mlContext.Transforms.Normalize(mode: NormalizingEstimator.NormalizerMode.MinMax, outputColumnName: nameof(Data.Features)))
                       .Append(mlContext.MulticlassClassification.Trainers.LightGbm(labelColumnName:nameof(Data.LabelIndex),  featureColumnName:nameof(Data.Features)))
                       .Append(mlContext.Transforms.Conversion.MapValueToKey(nameof(PredictedResult.PredictedLabelIndex), nameof(PredictedResult.PredictedLabel)))
                       .Append(mlContext.Transforms.CopyColumns(outputColumnName: nameof(PredictedResult.Scores), inputColumnName: DefaultColumnNames.Score));
            // Train the model.
            var model = pipeline.Fit(dataset.TrainSet);
  • What happened?
    Received error on log line:
Exception(s): LightGBM Error, code is -1, error message is 'bad allocation'.
Log: 18:20:52:[Source=LightGBMMulticlass; Loading data for LightGBM, Kind=Trace] Channel disposed. Elapsed 02:24:12.0168791.6010992.81611.0110556.
  • What did you expect?
    A error message that makes sense that I can understand what the issue is.

Source code / logs

No additional logs recorded,

Activity

  1. Ivanidzo4ka commented on Apr 15, 2019

    @Ivanidzo4ka
    Contributor

    This is something coming from LightGBM side.
    Somewhat related:
    lightgbm-org/LightGBM#1598

    If you can scramble small repro we can pass it to LightGBM folks

    Also not sure why you using:
    .Append(mlContext.Transforms.Conversion.MapValueToKey(nameof(PredictedResult.PredictedLabelIndex), nameof(PredictedResult.PredictedLabel)))
    Shouldn't it be KeyToValue instead?

  2. PeterPann23 commented on Apr 15, 2019

    @PeterPann23
    Author

    Hi,

    The code comes from
    Samples.Dynamic.Trainers.MulticlassClassification class LightGbm

    I added only normalisation and used NameOf instead of strings, see sample code on #3290

             var pipeline = mlContext.Transforms.Conversion.MapValueToKey("LabelIndex", "Label")
                            .Append(mlContext.MulticlassClassification.Trainers.LightGbm(labelColumnName: "LabelIndex"))
                            .Append(mlContext.Transforms.Conversion.MapValueToKey("PredictedLabelIndex", "PredictedLabel"))
                            .Append(mlContext.Transforms.CopyColumns("Scores", "Score"));
    
  3. PeterPann23 commented on Apr 16, 2019

    @PeterPann23
    Author

    Hi,

    text label to MapValueToKey results in to error:

    Schema mismatch for input column 'Label': expected KeyType, got Text
    Parameter name: inputSchema
    
  4. Ivanidzo4ka commented on Apr 16, 2019

    @Ivanidzo4ka
    Contributor

    That sample is definitely confusing. Good thing we rewriting it.

    So you have following pipeline:

    You have original data where labels are strings and features are vector of floats.
    You apply first MapValueToKey on top of Label column which would produce LabelIndex column with respectful keys. (I dunno who come up with Index part of name)
    That part is great and actually necessary since we multiclass works only on Key columns.
    So my comment wasn't about first MapValueToKey transform.

    By running trainer we will pop up two more columns PredictedLabel which is type Key and Score column which is vector of floats.

    Applying MapValueToKey on top of key type column is doable, but looks weird I don't understand what's the point of doing it.

    If you want to get original values back (from Key type for PredictedLabel) you either can use annotations

    engine.OutputSchema[nameof(IrisPrediction.PredictedLabel)].GetKeyValues(ref keys);

    or use MapKeyToValue
    .Append(mlContext.Transforms.Conversion.MapKeyToValue("PredictedLabelInOriginalType", "PredictedLabel"))

  5. PeterPann23 commented on Apr 16, 2019

    @PeterPann23
    Author

    cool, thanks for the update

  6. ghost locked as resolved and limited conversation to collaborators on Mar 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions