`csv` does not round-trip for `complex` numbers

[sobolevn]

Reproduction depends on the quoting.

QUOTE_ALL

import csv
data = [1, 1j]
with open('example.csv', 'w') as f:
    writer = csv.writer(f, quoting=csv.QUOTE_ALL)
    writer.writerow(data)

with open('example.csv') as f:
    contents = f.read()
    print(contents)  # 1,1j
    reader = csv.reader(contents, quoting=csv.QUOTE_ALL)
    print(list(reader))  # [['1'], ['', ''], ['1'], ['j'], []]

In this case, we don't have our data back, but at least it does not raise.

QUOTE_NONNUMERIC

Qouting docs:

No automatic data type conversion is performed unless the QUOTE_NONNUMERIC format option is specified (in which case unquoted fields are transformed into floats)

import csv
data = [1, 1j]
with open('example.csv', 'w') as f:
    writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
    writer.writerow(data)

with open('example.csv') as f:
    contents = f.read()
    print(contents)  # 1,1j
    reader = csv.reader(contents, quoting=csv.QUOTE_NONNUMERIC)
    print(list(reader))

# Traceback (most recent call last):
#   File "/Users/sobolev/Desktop/cpython/ex.py", line 11, in <module>
#     print(list(reader))
#           ^^^^^^^^^^^^
# ValueError: could not convert string to float: 'j'

In this case, it raises an error, while trying to convert 1j to float.

Current docs / tests

I cannot find any mentions of complex numbers in tests or docs for csv.

Solutions?

1. We can say that it works as expected: with this strange QUOTE_ALL formatting and QUOTE_NONNUMERIC exception. Add a test case for it and forget about it
2. We can try to quote complex as string: in this case it will be treated as "1j". I think it is much better, because it will allow users to convert this value to complex manually
3. Forbid writting complex numbers. However, I don't think it is a path we should go

[pochmann]

I suggest you first fix your bug. Give csv.reader the file object, not the file contents as a string.

[pochmann]

Those two programs are identical. You're not using QUOTE_NONNUMERIC.

[sobolevn]

@pochmann thanks!

I suggest you first fix your bug. Give csv.reader the file object, not the file contents as a string.

It does not really matter. See:

» ./python.exe
Python 3.12.0a0 (heads/main:ff173ed2f6, Oct 20 2022, 14:06:56) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import csv
>>> data = [1, 1j]
>>> with open('example.csv', 'w') as f:
...     writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
...     writer.writerow(data)
... 
6
>>> with open('example.csv') as f:
...     reader = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
...     print(list(reader))
... 
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
ValueError: could not convert string to float: '1j'

Those two programs are identical. You're not using QUOTE_NONNUMERIC.

This was a copy-paste error, updated.

[akulakov]

Only addressing the first example, with quoting=csv.QUOTE_ALL or with default quoting, I do get the original data back as expected (on latest dev version):

with open('a.csv') as f:
    reader = csv.reader(f, quoting=csv.QUOTE_ALL)
    for r in reader:
        print(r)
# ['1', '1j']
# ['2', '2j']

So with complex numbers you would need to complex(r[1]) when reading data.

If there was no way to get the original data back, of course it would be more alarming.

[skirpichev]

@sobolevn, this doesn't work for floats as well. Or did I miss something?

>>> import csv
... data = [3.14]
... with open('example.csv', 'w') as f:
...     writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
...     writer.writerow(data)
...     
6
>>> with open('example.csv') as f:
...     contents = f.read()
...     print(contents)
...     reader = csv.reader(contents, quoting=csv.QUOTE_NONNUMERIC)
...     print(list(reader))
...     
3.14

Traceback (most recent call last):
  File "<python-input-1>", line 5, in <module>
    print(list(reader))
          ~~~~^^^^^^^^
ValueError: could not convert string to float: '.'

[serhiy-storchaka]

gh-155097 (PR #155099) adds a converter parameter to csv.reader which replaces the hard-coded float(). Since str(1j) is already written unquoted, complex numbers round-trip with:

csv.reader(f, quoting=csv.QUOTE_NONNUMERIC, converter=lambda i, s: complex(s))

The converter is called as converter(index, field), so converter=complex does not work — the index would be taken as the real part and the field as imag.

The QUOTE_ALL half of the report is unrelated and works as documented: in that mode the reader converts nothing, so every field is returned as a string.