Skip to content

cf-python or upstream issue with h5py? Can't install new cf-python #899

Closed
Closed
cf-python or upstream issue with h5py? Can't install new cf-python#899
Labels
bug?

Description

@bnlawrence
bnlawrence
opened on Oct 16, 2025

This is in a brand new python mamba install with cf-python installed via a pip dependency.
Can't run cf.environment :-)

Is this a problem with what we have pinned, or is this a problem with the h5netcdf /pyfive status right now? Either way we probably need to do something?

ipython
Python 3.11.14 | packaged by conda-forge | (main, Oct 13 2025, 14:29:50) [Clang 19.1.7 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.6.0 -- An enhanced Interactive Python. Type '?' for help.
Tip: `?` alone on a line will brings up IPython's help
Cmd click to launch VS Code Native REPL

In [1]: import cf
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[1], line 1
----> 1 import cf

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/cf/__init__.py:120
    114         raise RuntimeError(
    115             f"Bad packaging version: cf requires packaging>={_minimum_vn}. "
    116             f"Got {packaging.__version__} at {packaging.__file__}"
    117         )
    119 try:
--> 120     import cfdm
    121 except ImportError as error1:
    122     raise ImportError(_error0 + str(error1))

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/cfdm/__init__.py:93
     91 # Check the version of h5netcdf
     92 try:
---> 93     import h5netcdf
     94 except ImportError as error1:
     95     raise ImportError(_error0 + str(error1))

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/h5netcdf/__init__.py:16
     11 except Exception:
     12     # Local copy or not installed with setuptools.
     13     # Disable minimum version checks on downstream libraries.
     14     __version__ = "999"
---> 16 from .core import CompatibilityError, Dimension, File, Group, Variable  # noqa

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/h5netcdf/core.py:9
      6 from collections import ChainMap, Counter, OrderedDict, defaultdict
      7 from collections.abc import Mapping
----> 9 import h5py
     10 import numpy as np
     11 from packaging import version

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/h5py/__init__.py:45
     36     _warn(("h5py is running against HDF5 {0} when it was built against {1}, "
     37            "this may cause problems").format(
     38             '{0}.{1}.{2}'.format(*version.hdf5_version_tuple),
     39             '{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
     40     ))
     43 _errors.silence_errors()
---> 45 from ._conv import register_converters as _register_converters, \
     46                    unregister_converters as _unregister_converters
     47 _register_converters()
     48 atexit.register(_unregister_converters)

File h5py/_conv.pyx:1, in init h5py._conv()

File h5py/h5r.pyx:1, in init h5py.h5r()

File h5py/h5p.pyx:1, in init h5py.h5p()

File h5py/h5t.pyx:152, in init h5py.h5t()

File h5py/h5t.pyx:86, in h5py.h5t.lockid()

File h5py/h5t.pyx:55, in h5py.h5t.typewrap()

RuntimeError: Not a datatype (not a datatype)

To report a bug, please provide:

  • A description of the issue with, if possible,

    • what you expected to happen,
    • the steps needed to reproduce it,
    • any traceback information.

  • The version of the software and the environment in which you are encountering an issue. The output of cf.environment(paths=False) is useful for this.

Activity

  1. bnlawrence
    added
    bug?
    on Oct 16, 2025

  2. davidhassell commented on Oct 16, 2025

    @davidhassell
    davidhassell
    on Oct 16, 2025
    Collaborator

    Thanks, Bryan - which versions of netCDF4 and libhdf5 are you using?

  3. bnlawrence commented on Oct 16, 2025

    @bnlawrence
    bnlawrence
    on Oct 16, 2025
    ContributorAuthor

    Python netCDF4 version: 1.7.3
    Linked against libnetcdf version: 4.9.3
    Linked against HDF5 version: 1.14.6

  4. davidhassell commented on Oct 16, 2025

    @davidhassell
    davidhassell
    on Oct 16, 2025
    Collaborator

    Ta. Known problems (arising in last day or two) with the latest combination of netCDF4 linked to HDF5 1.14.6. Easiest bet is to pin netCDF4==1.7.2.

  5. bnlawrence commented on Oct 16, 2025

    @bnlawrence
    bnlawrence
    on Oct 16, 2025
    ContributorAuthor

    where?

  6. davidhassell commented on Oct 16, 2025

    @davidhassell
    davidhassell
    on Oct 16, 2025
    Collaborator

    something like pip install netCDF4==1.7.2 might do it (or some conda-like equivalent)

  7. bnlawrence
    changed the title [-]cf-python or upstream issue with pyfive? Can't install new cf-python[/-] [+]cf-python or upstream issue with h5py? Can't install new cf-python[/+] on Oct 16, 2025

  8. bnlawrence commented on Oct 16, 2025

    @bnlawrence
    bnlawrence
    on Oct 16, 2025
    ContributorAuthor

    Sadly, that doesn't look it worked:

    Installing collected packages: netCDF4
  Attempting uninstall: netCDF4
    Found existing installation: netCDF4 1.7.3
    Uninstalling netCDF4-1.7.3:
      Successfully uninstalled netCDF4-1.7.3
Successfully installed netCDF4-1.7.2
(s3v_25) bnl28@MX6H7D9YGP s3v % ipython
Python 3.11.14 | packaged by conda-forge | (main, Oct 13 2025, 14:29:50) [Clang 19.1.7 ]
Type 'copyright', 'credits' or 'license' for more information
IPython 9.6.0 -- An enhanced Interactive Python. Type '?' for help.
Tip: Use `--theme`, or the `%colors` magic to change IPython's themes and colors.
Cmd click to launch VS Code Native REPL

In [1]: import cf
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[1], line 1
----> 1 import cf

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/cf/__init__.py:120
    114         raise RuntimeError(
    115             f"Bad packaging version: cf requires packaging>={_minimum_vn}. "
    116             f"Got {packaging.__version__} at {packaging.__file__}"
    117         )
    119 try:
--> 120     import cfdm
    121 except ImportError as error1:
    122     raise ImportError(_error0 + str(error1))

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/cfdm/__init__.py:93
     91 # Check the version of h5netcdf
     92 try:
---> 93     import h5netcdf
     94 except ImportError as error1:
     95     raise ImportError(_error0 + str(error1))

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/h5netcdf/__init__.py:16
     11 except Exception:
     12     # Local copy or not installed with setuptools.
     13     # Disable minimum version checks on downstream libraries.
     14     __version__ = "999"
---> 16 from .core import CompatibilityError, Dimension, File, Group, Variable  # noqa

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/h5netcdf/core.py:9
      6 from collections import ChainMap, Counter, OrderedDict, defaultdict
      7 from collections.abc import Mapping
----> 9 import h5py
     10 import numpy as np
     11 from packaging import version

File ~/mambaforge/envs/s3v_25/lib/python3.11/site-packages/h5py/__init__.py:45
     36     _warn(("h5py is running against HDF5 {0} when it was built against {1}, "
     37            "this may cause problems").format(
     38             '{0}.{1}.{2}'.format(*version.hdf5_version_tuple),
     39             '{0}.{1}.{2}'.format(*version.hdf5_built_version_tuple)
     40     ))
     43 _errors.silence_errors()
---> 45 from ._conv import register_converters as _register_converters, \
     46                    unregister_converters as _unregister_converters
     47 _register_converters()
     48 atexit.register(_unregister_converters)

File h5py/_conv.pyx:1, in init h5py._conv()

File h5py/h5r.pyx:1, in init h5py.h5r()

File h5py/h5p.pyx:1, in init h5py.h5p()

File h5py/h5t.pyx:152, in init h5py.h5t()

File h5py/h5t.pyx:86, in h5py.h5t.lockid()

File h5py/h5t.pyx:55, in h5py.h5t.typewrap()

File h5py/defs.pyx:3564, in h5py.defs.H5Tget_class()

RuntimeError: Unspecified error in H5Tget_class (return value <0)

In [2]:

  9. davidhassell commented on Oct 16, 2025

    @davidhassell
    davidhassell
    on Oct 16, 2025
    Collaborator

    Oh. I think it might be less netCDF4, and more the libhdf5 side (we discussed this in our data meeting). @valeriupredoi can you advise?

  10. valeriupredoi commented on Oct 16, 2025

    @valeriupredoi
    valeriupredoi
    on Oct 16, 2025

    @bnlawrence get your netCDF4 from conda forge with mamba install -c conda-forge netcdf4 but that in an environment with no netcdf4 installed from PyPI since then conda will say all packages installed

  11. valeriupredoi commented on Oct 16, 2025

    @valeriupredoi
    valeriupredoi
    on Oct 16, 2025

    @davidhassell netcdf4=1.7.2 is also borked from PyPI (with the 27 wheel rebuild), see your GHA https://github.com/NCAS-CMS/cf-python/actions/runs/18514295542/job/52761573174

  12. valeriupredoi commented on Oct 16, 2025

    @valeriupredoi
    valeriupredoi
    on Oct 16, 2025

    @davidhassell netcdf4=1.7.2 is also borked from PyPI (with the 27 wheel rebuild), see your GHA https://github.com/NCAS-CMS/cf-python/actions/runs/18514295542/job/52761573174

    I think that's solved now with the manylinux 28 wheel build BTW

  13. bnlawrence commented on Oct 16, 2025

    @bnlawrence
    bnlawrence
    on Oct 16, 2025
    ContributorAuthor

    @bnlawrence get your netCDF4 from conda forge with mamba install -c conda-forge netcdf4 but that in an environment with no netcdf4 installed from PyPI since then conda will say all packages installed

    Can confirm that works.

  14. bnlawrence
    closed this as completedon Oct 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug?

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions