Plotly express not compatible with pandas NAType/pandas.NA/pd.NA values

Plotly express functions fall over when given dataframes containing pd.NA values. I think these should have the same behaviour as None values.

Example:

# This works
px.scatter(pd.DataFrame([1, 2, 3, None, 5]))

# This results in "TypeError: Object of type NAType is not JSON serializable"
px.scatter(pd.DataFrame([1, 2, 3, None, 5]).convert_dtypes())

Thanks for pointing that out! We'll try to get it fixed soon :)

Note to self: will need to look at

plotly.py/packages/python/plotly/_plotly_utils/utils.py

Lines 168 to 178 in b34bd6d

    
           @staticmethod 
        
           def encode_as_pandas(obj): 
        
               """Attempt to convert pandas.NaT""" 
        
               pandas = get_module("pandas", should_load=False) 
        
               if not pandas: 
        
                   raise NotEncodable 
        
               if obj is pandas.NaT: 
        
                   return None 
        
               else: 
        
                   raise NotEncodable

plotly.py/packages/python/plotly/plotly/io/_json.py

Lines 507 to 526 in b34bd6d

    
           if pd is not None: 
        
               if obj is pd.NaT: 
        
                   return None 
        
               elif isinstance(obj, (pd.Series, pd.DatetimeIndex)): 
        
                   if numpy_allowed and obj.dtype.kind in ("b", "i", "u", "f"): 
        
                       return np.ascontiguousarray(obj.values) 
        
                   elif obj.dtype.kind == "M": 
        
                       if isinstance(obj, pd.Series): 
        
                           dt_values = obj.dt.to_pydatetime().tolist() 
        
                       else:  # DatetimeIndex 
        
                           dt_values = obj.to_pydatetime().tolist() 
        
                       if not datetime_allowed: 
        
                           # Note: We don't need to handle dropping timezones here because 
        
                           # numpy's datetime64 doesn't support them and pandas's tz_localize 
        
                           # above drops them. 
        
                           for i in range(len(dt_values)): 
        
                               dt_values[i] = dt_values[i].isoformat() 
        
                       return dt_values

This simple addition worked for me when i used px.timeline #3336. Note that I'm a plotly newbie.

stolen from https://github.com/GoogleCloudPlatform/covid-19-open-data/pull/265/files :)

#3705 partially fixed this issue. The issue is fixed when Python's json is used, but the problem persists for orjson.

import orjson  # assert that orjson is available

plotly.io.json.config.default_engine = 'json'
px.scatter(pd.DataFrame([1, 2, 3, None, 5]).convert_dtypes()).to_json()  # works
plotly.io.json.config.default_engine = 'auto'  # or `orjson`
px.scatter(pd.DataFrame([1, 2, 3, None, 5]).convert_dtypes()).to_json()  # fails

@avm19

thank you for taking a look at this @avm19 .

@gvwilson

Hi - we are tidying up stale issues and PRs in Plotly's public repositories so that we can focus on things that are still important to our community. Since this one has been sitting for a while, I'm going to close it; if it is still a concern, please add a comment letting us know what recent version of our software you've checked it with so that I can reopen it and add it to our backlog. If you'd like to submit a PR, we'd be happy to prioritize a review, and if it's a request for tech support, please post in our community forum. Thank you - @gvwilson

Navigation Menu

Repository navigation

Plotly express not compatible with pandas NAType/pandas.NA/pd.NA values #3253

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	@staticmethod
	def encode_as_pandas(obj):
	"""Attempt to convert pandas.NaT"""
	pandas = get_module("pandas", should_load=False)
	if not pandas:
	raise NotEncodable

	if obj is pandas.NaT:
	return None
	else:
	raise NotEncodable

	if pd is not None:
	if obj is pd.NaT:
	return None
	elif isinstance(obj, (pd.Series, pd.DatetimeIndex)):
	if numpy_allowed and obj.dtype.kind in ("b", "i", "u", "f"):
	return np.ascontiguousarray(obj.values)
	elif obj.dtype.kind == "M":
	if isinstance(obj, pd.Series):
	dt_values = obj.dt.to_pydatetime().tolist()
	else: # DatetimeIndex
	dt_values = obj.to_pydatetime().tolist()

	if not datetime_allowed:
	# Note: We don't need to handle dropping timezones here because
	# numpy's datetime64 doesn't support them and pandas's tz_localize
	# above drops them.
	for i in range(len(dt_values)):
	dt_values[i] = dt_values[i].isoformat()

	return dt_values

Plotly express not compatible with pandas NAType/pandas.NA/pd.NA values #3253

Description

Activity

nicolaskruchten commented on Jun 18, 2021

nicolaskruchten commented on Jun 18, 2021

astafan8 commented on Aug 5, 2021

avm19 commented on Jan 17, 2024

Coding-with-Adam commented on Jan 18, 2024

gvwilson commented on Jul 11, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions