Skip to content

Wheel generation. #3

Open
Open

Description

@4ment

Generate Python wheels with Arrow/Parquet dependencies

Description

Create distributable Python wheels for cpplink.

The Python bindings themselves are implemented, but generating a wheel is currently blocked by the C++ linking stage. cpplink depends on Apache Arrow and Parquet, and the wheel build needs to correctly resolve and package/link these native dependencies.

The main challenge is determining how Arrow and Parquet should be handled when building the wheel, particularly across different platforms.

Goals

  • Build a Python wheel containing the cpplink Python bindings.
  • Resolve the C++ linking against Arrow and Parquet correctly during the wheel build.
  • Determine whether Arrow/Parquet libraries should be bundled with the wheel or provided as external dependencies.
  • Ensure the resulting wheel can be installed into a clean Python environment.
  • Test the installed package by importing cpplink and exercising Parquet functionality.

Areas to investigate

  • CMake integration with Arrow and Parquet.
  • Correct propagation of Arrow/Parquet include directories and libraries to the Python extension.
  • Runtime library discovery (rpath, shared libraries, etc.).
  • Wheel packaging of native libraries if required.
  • Platform-specific behaviour on macOS and Windows.
  • Interaction with pybind11 and the Python extension module.
  • Whether Arrow/Parquet should instead be provided through Conda dependencies.

Acceptance criteria

  • A wheel can be generated successfully from a clean build.
  • The wheel installs successfully into a clean Python environment.
  • import cpplink works without requiring a local development build.
  • Arrow/Parquet symbols are correctly resolved at build and runtime.
  • Parquet read/write functionality works from Python.
  • The dependency strategy for Arrow/Parquet is documented.
  • The approach can be extended to Windows wheels.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions