Skip to content

Handle serializing dicts with ints and strings #3380

Description

@chriddyp

The current to_json_plotly doesn't handle dicts with integers and strings as keys. This used to work before we set sort_keys=True.

This is relatively common in Dash.

Here is a MWE:

import pandas as pd
from plotly.io.json import to_json_plotly

df = pd.DataFrame({'a': [0], 1: [0]})
to_json_plotly(df.to_dict('records'))
TypeError: '<' not supported between instances of 'int' and 'str'

Here is the issue in the Python tracker: https://bugs.python.org/issue25457 re sort_keys=True

Originally reported in https://community.plotly.com/t/dash-2-0-0-migration-error/56478/7

Activity

  1. nicolaskruchten commented on Sep 13, 2021

    @nicolaskruchten
    Contributor

    @jonmmease is this something you could take a look at please? There's going to be some ambiguity if the same dict has 1 and "1" as keys...

  2. nicolaskruchten commented on Sep 13, 2021

    @nicolaskruchten
    Contributor

    Admittedly, that's an ambiguity that JSON imposes.

  3. jonmmease commented on Sep 14, 2021

    @jonmmease
    Contributor

    FWIW, this can be reproduced without plotly.py

    import json
    json.dumps({'a': 0, 1: 0}, sort_keys=True)
    TypeError: '<' not supported between instances of 'int' and 'str'
    

    The reason that we're hitting this now is that we switched to using sort_keys=True during the orjson refactor.

    As @chriddyp pointed out, there is a 2+ year long discussion in https://bugs.python.org/issue25457 on whether this is actually a bug in the Python 3 json module.

    Maybe it was a mistake to set sort_keys=True. This is the behavior of orjson, and at the time it seemed reasonable to align the output across json engines 🙂

    I don' t see a way to address this from within our custom PlotlyJSONEncoder, so I see two options:

    • Remove sort_keys=True, adjusting tests to accept that the json engines will not produce identical results
    • Try json.dumps and catch the exception. If the exception is raised, clone the input dict and modify it to convert non-string keys to strings before running json.dumps again.

    Thoughts @nicolaskruchten ?

  4. nicolaskruchten commented on Sep 15, 2021

    @nicolaskruchten
    Contributor

    Hmmm yeah, it kinda feels like "Remove sort_keys=True, adjusting tests to accept that the json engines will not produce identical results" is the safer route... basically getting folks to opt-in to orjson and any related constraints. Sorry!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugsomething broken

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions