Skip to content

Microsoft.Data.Analysis DataFrameColumns.SetName method is weird #6129

Closed

Description

@olavt

.Net Core 3.1
Microsoft.Data.Analysis Nuget package version: 0.19.1

This would be the logical way to call the method:

dataFrame.Columns["Date_left"].SetName("Date");

But, this does not result in the column being properly renamed.

The method needs to be called like this in order to work properly:

dataFrame.Columns["Date_left"].SetName("Date", dataFrame);

Here's a program to reproduce the issue:

using Microsoft.Data.Analysis;
using System;
using System.Linq;

namespace TestDataFRame
{
    internal class Program
    {
        static void Main(string[] args)
        {
            DateTime?[] dates1 = { new DateTime(2022, 03, 01), new DateTime(2022, 03, 02), new DateTime(2022, 03, 03) };
            double?[] closePrices = { 10.5, 12.4, 11.3 };

            DateTime?[] dates2 = { new DateTime(2022, 03, 01), new DateTime(2022, 03, 02), new DateTime(2022, 03, 03), new DateTime(2022, 03, 04) };
            double[] shortPercentages = { 2.34, 2.36, 3.01, 3.04 };

            DataFrame dataFrame1 = new DataFrame();
            dataFrame1.Columns.Add(new PrimitiveDataFrameColumn<DateTime>("Date", dates1));
            dataFrame1.Columns.Add(new DoubleDataFrameColumn("ClosePrice", closePrices));

            var numbers1 = dataFrame1.Columns.GetDoubleColumn("ClosePrice").ToArray();

            DataFrame dataFrame2 = new DataFrame();
            dataFrame2.Columns.Add(new PrimitiveDataFrameColumn<DateTime>("Date", dates2));
            dataFrame2.Columns.Add(new DoubleDataFrameColumn("ShortPercentage", shortPercentages));

            var numbers2 = dataFrame2.Columns.GetDoubleColumn("ShortPercentage").ToArray();

            DataFrame dataFrame = dataFrame1.Merge<DateTime>(dataFrame2, "Date", "Date", joinAlgorithm: JoinAlgorithm.Left);
            dataFrame.Columns["Date_left"].SetName("Date");

            var dates = dataFrame.Columns.GetPrimitiveColumn<DateTime>("Date").ToArray();
        }
    }
}

Activity

  1. michaelgsharp commented on Mar 18, 2022

    @michaelgsharp
    Contributor

    Looks like the columns themselves don't have a reference to the parent DataFrame, hence this behavior. I think the way to fix this would be to have the column have a reference to the DataFrame so it can perform the update as you point out. I dont think it would be a big change. @luisquintanilla

  2. michaelgsharp commented on Mar 18, 2022

    @michaelgsharp
    Contributor

    Ok, the other option (or we could do both), is that dataFrame.Columns does have a SetColumnName method as well, it just for some reason requires you to pass in the whole column SetColumnName(DataFrameColumn column, string newName). We could just overload that method to just take the name of the current column and the name you want to set it to SetColumnName(string curName, string newName). Then you could just call dataFrame.Columns.SetColumnName("cur", "new");. Thoughts?

  3. luisquintanilla commented on Mar 29, 2022

    @luisquintanilla
    Contributor

    Thanks for reporting this issue @olavt.

    @michaelgsharp Thanks for looking into this and providing those solutions. Tracking this issue as part of DataFrame improvements.

  4. added this to the ML.NET Future milestone on Apr 11, 2022
  5. chrisxfire commented on Aug 10, 2022

    @chrisxfire

    Note that, because SetName has a default value of null for the DataFrame parameter, if a user tries dataFrame["oldColumnName"]("newColumnName"), not only will this fail, but it will also not throw an exception, resulting in a bug that is hard to catch.

    You can guess who the user is... 😀

  6. ghost added
    in-prA pull request is open to resolve this issue
    on May 14, 2023
  7. ghost removed
    in-prA pull request is open to resolve this issue
    on Jul 6, 2023
  8. ghost locked as resolved and limited conversation to collaborators on Aug 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions