Skip to content

ENH: Ingest ITKTotalVariation into Modules/Filtering (with ITKproxTV third-party module) - #6381

Merged
hjmjohnson merged 61 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:ingest-TotalVariation
Jun 6, 2026
Merged

hjmjohnson merged 61 commits into
InsightSoftwareConsortium:mainfrom
hjmjohnson:ingest-TotalVariation

Conversation

@hjmjohnson

Copy link
Copy Markdown
Member

Ingests the TotalVariation remote module into Modules/Filtering/TotalVariation, and adds a new Modules/ThirdParty/proxTV third-party module (DCMTK-style: ExternalProject-built at build time from the ISC proxTV fork, EXCLUDE_FROM_DEFAULT, no vendored source). Builds and tests under C++17 against current main. Part of #6160; consumes blowekamp's ITKTotalVariation#57 (ISC-forked proxTV).

Local validation: Module_TotalVariation=ON configures, proxTV builds via ExternalProject under C++17 (Eigen backend), and 3/3 tests pass including itkProxTVImageFilterTest. pre-commit run --all-files is clean.

proxTV integration model ("like DCMTK")
  • Modules/ThirdParty/proxTV/ holds only CMake glue — proxTV source is not vendored into ITK git. It is git-cloned + built at build time via ExternalProject from InsightSoftwareConsortium/proxTV (tag pinned in proxTVGitTag.cmake).
  • EXCLUDE_FROM_DEFAULT: only configured/built when a dependent module (TotalVariation) is enabled. ITK_USE_SYSTEM_proxTV selects an external build.
  • proxTV is built with the Eigen backend (proxTV_USE_LAPACK=OFF) against ITK's internal Eigen3, under ITK's C++17 standard (-DCMAKE_CXX_STANDARD=17). The proxTV fork needed no change for C++17.
  • The imported target carries IMPORTED_LOCATION, the source src/ include dir, NOMATLAB (guards proxTV's mex.h), and Threads::Threads.
  • Library + public headers are installed into the ITK install tree, with build- and install-tree EXPORT_CODE so external consumers of an installed ITK resolve proxTV and TVopt.h.
Commits
  1. ENH: Ingest merge — merge-preserving filter-repo (122→56 commits, upstream merge topology + authorship preserved).
  2. COMP: Add ITKproxTV third-party module; wire TotalVariation in-tree.
  3. COMP: Remove TotalVariation.remote.cmake.
  4. ENH: Enable Module_TotalVariation in configure-ci.
  5. DOC: Module README.
  6. STYLE: gersemi-format ingested test CMakeLists.
  7. BUG: Correct ProxTVImageFilter OutputImageType (TOutputImage, not TInputImage); explicit size→int cast; header-guard fix.
Reviewer notes
  • Install/packaging: install-tree consumption (an external project compiling against an installed ITK with TotalVariation) is wired but only validated via the generated cmake_install.cmake; a real make install + downstream-consume pass in CI would be a good confirmation.
  • Python wrapping was not exercised locally (non-Python build tree); CI covers it. The .wrap covers 2D WRAP_ITK_REAL.
  • The ingested filter test runs the filter but does not regression-check output values (upstream --compare baseline was commented out); not added here.

Per ingest workflow, upstream ITKTotalVariation archival is a separate post-merge follow-up.

phcerdan and others added 30 commits March 27, 2018 12:41
Remove some placeholders
find_package(proxTV REQUIRED CONFIG)
Wrapped two methods:

DR2_TV for 2D
and the general PD_TV for 3D

Weights (ie. lambdas) and norms can be changed by the user.
ExternalProject or FetchContent?
Uses ${_proxTV_lib} to handle both cases:
- using find_package(proxTV) -> proxTV::prox
- using add_subdirectory(proxTV_folder) -> proxTV

Export the targets to the module targets file:
TotalVariationTargets.cmake
The nice thing about targets, is that we only have to worry about the
proxTV targets, all its dependencies, even if built internally, are
handled.

Note that install is not handled.
TODO: Manage Eigen3
Options: system or internal with ITKEigen3
```cmake
set(_internal_cmake_eigen3)
list(GET ITKEigen3_INCLUDE_DIRS 0 _internal_cmake_eigen3)
set(Eigen3_DIR "${_internal_cmake_eigen3}/itkeigen")
```
Also re-arranges to avoid duplication
Add itk-module-init.cmake
WIP: Remove debug CMake messages

SWIG is not reading the TotalVariationTargets.cmake file, where proxTV
library is set, with include dirs, interface libraries and compile
definitions.
This creates erorrs, the first sign is missing the INCLUDE_DIRSs.
(TVopt.h not found).
This can be solved hacking the INCLUDE_DIRS of the ITK Module TotalVariation_INCLUDE_DIRS.

Once that's fixed, the next error is missing mex.h file, this is because
the compile definition NO_MATLAB is not applied.

In Eigen3 I faced the same problems, but because it is header only,
fixing INCLUDE_DIRS and adding the compile definitions in the
Eigen3-ITK Header (itkEigen3.h) was enough.

In this case, the compile definitions of proxTV (NO_MATLAB, USE_LAPACK=0)
are ignored.
COMP: CMake, add INCLUDE_DIRS for swig to work
BUG: Add compile definitions for wrapping to work
Set TotalVariation_LIBRARIES to proxTV directly
Addresses Issue InsightSoftwareConsortium#26
Copy image infromation from input
…r/Issue26-ImageAxes-Change

FIX: Image Axes Change

@dzenanz dzenanz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good after a relatively quick look.

@blowekamp

Copy link
Copy Markdown
Member

@hans May I take over this work?

The modern CMake usage now available in ITK is to use the interface modules as opposed to the legacy cmake variables. Changes were made in the proxTV library fork to support this. It would be good to use the modern best practices with new code integration opposed to the older approaches (that the agents may be picking up on).

Please let me know if I can continue this effort with the modern approach.

@hjmjohnson

Copy link
Copy Markdown
Member Author

May I take over this work?

@blowekamp YES! We're very close to closing issue #6160 for remote module ingestions. Once the open PR's are included, we will be able to close that issue!

My only request is that (a) the improvements are made in the next few days, or (b) we include the minimally working ingestion as is, and follow up with improvements.

Hans

@blowekamp
blowekamp self-requested a review June 4, 2026 17:27
@blowekamp
blowekamp marked this pull request as draft June 4, 2026 17:27

@blowekamp blowekamp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will be taking over this PR. Mostly to work on the proxTV integration as a third party.

@hjmjohnson

Copy link
Copy Markdown
Member Author

I will be taking over this PR. Mostly to work on the proxTV integration as a third party.

Awesome! Thank you.

@blowekamp
blowekamp force-pushed the ingest-TotalVariation branch from 2ca7e11 to cf0be3d Compare June 4, 2026 19:31

@dzenanz dzenanz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A significant simplification.

Comment thread Modules/ThirdParty/proxTV/CMakeLists.txt
itk_module_impl()
else()
# 3.23 Required for proxTV FILE_SETS, which allows for implicit installation of header files with the library target.
cmake_minimum_required(VERSION 3.23.0)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ubuntu 24.04 comes with CMake 3.28.3, and Ubuntu 26.04 was released recently. Should we bump general CMake version requirement, instead of just here? Or wait for ITK 6.1 to do that?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hjmjohnson What do you think of updating ITK's CMake version? May make some of the fetch content work easier if we are going down that route further.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that getting modern CMake versions is so easy for all developers that we should be more aggressive in updating the versions. New versions of cmake can be easily downloaded via pixi, uv, simple shell commands,.

CMake version 3.28.3 seems like a great floor.

My one concern is the Slicer support. That codebase is already littered with many conditionals, so they may be receptive to bumping the cmake minimum version as well:

┌─[johnsonhj@hj] - [~/src/Slicer] - [2026-06-05 07:34:18]
└─[0] rg CMAKE_VERSION                                                                                                          (main|✔)
Extensions/CMake/SlicerBlockBuildPackageAndUploadExtensions.cmake
230:  if(CMAKE_VERSION GREATER_EQUAL "3.28")

CMakeLists.txt
6:# OLDEST_VALIDATED_POLICIES_VERSION and CMAKE_VERSION (used for this build)
8:# between the OLDEST_VALIDATED_POLICIES_VERSION and CMAKE_VERSION.
17:if("${CMAKE_VERSION}" VERSION_EQUAL "3.21.0")
18:  message(FATAL_ERROR "CMake version is ${CMAKE_VERSION} and using CMake==3.21.0 is not supported.\nSee https://gitlab.kitware.com/cmake/cmake/-/issues/22476")
21:if("${CMAKE_VERSION}" VERSION_GREATER_EQUAL "3.25.0" AND "${CMAKE_VERSION}" VERSION_LESS_EQUAL "3.25.2")
22:  message(FATAL_ERROR "CMake version is ${CMAKE_VERSION} and using CMake >=3.25.0,<=3.25.2 is not supported.\nSee https://gitlab.kitware.com/cmake/cmake/-/issues/24567")
912:if("${CMAKE_VERSION}" VERSION_GREATER_EQUAL "3.29.0")

CMake/SlicerMacroBuildScriptedCLI.cmake
47:    if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.20")

CMake/ExternalProjectDependency.cmake
469:  if(NOT CMAKE_VERSION VERSION_LESS "3.16")
487:  if(NOT CMAKE_VERSION VERSION_LESS "3.16")
516:  if(CMAKE_VERSION VERSION_GREATER "3.0")
520:  if(CMAKE_VERSION VERSION_EQUAL "3.4" OR CMAKE_VERSION VERSION_GREATER "3.4")
528:  if(CMAKE_VERSION VERSION_EQUAL "3.24" OR CMAKE_VERSION VERSION_GREATER "3.24")
532:  if(CMAKE_VERSION VERSION_EQUAL "3.28" OR CMAKE_VERSION VERSION_GREATER "3.28")

SuperBuild/External_python.cmake
60:  if(CMAKE_VERSION VERSION_GREATER_EQUAL "3.24")

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blowekamp At a worst case, if building ITKTtotalVariaion, REQUIRE cmake 3.28 or greater as a stopgap until the rest of the community hits the same floor.

@blowekamp
blowekamp marked this pull request as ready for review June 5, 2026 14:08
@blowekamp

Copy link
Copy Markdown
Member

@hjmjohnson I think I am done with the PR. It looks like it needs to be rebased due to a conflict, but with the merge and the orphan branch a simple rebase will now work.

@hjmjohnson

Copy link
Copy Markdown
Member Author

@blowekamp I will rebase this morning.

hjmjohnson and others added 5 commits June 5, 2026 10:52
Brings TotalVariation from a configure-time remote fetch into the ITK
source tree at Modules/Filtering/TotalVariation/ using the v4 ingestion
pipeline (whitelist filter-repo + per-commit clang-format + black +
commit-prefix sanitization).

Upstream repo:  https://github.com/InsightSoftwareConsortium/ITKTotalVariation.git
Upstream tip:   a776d94799b7204d19b1c69c331458ed59fd0a3f
Ingest date:    2026-06-02
Whitelist:      default.list

Per-commit transforms applied across all 55 commits:
  - filter-repo --paths-from-file (whitelist)
  - filter-repo --to-subdirectory-filter Modules/Filtering/TotalVariation
  - clang-format -style=file (ITK main's .clang-format) for *.cxx/.h/.hxx/...
  - black for *.py
  - heuristic ITK prefix added to commit subjects without one

Merge topology preserved: 33 -> 10 merge(s).

Primary author: Pablo Hernandez-Cerdan <pablo.hernandez.cerdan@outlook.com>

Co-authored-by: Bradley Lowekamp <blowekamp@mail.nih.gov>
Co-authored-by: Bryn Lloyd <lloyd@itis.swiss>
Co-authored-by: Dženan Zukić <dzenan.zukic@kitware.com>
Co-authored-by: Hans Johnson <hans-johnson@uiowa.edu>
Co-authored-by: Jean-Christophe Fillion-Robin <jchris.fillionr@kitware.com>
Co-authored-by: Jon Haitz Legarreta Gorroño <jon.haitz.legarreta@gmail.com>
Co-authored-by: Mathew Seng <mathewseng@gmail.com>
Co-authored-by: Matt McCormick <matt.mccormick@kitware.com>
Co-authored-by: Matt McCormick <matt@mmmccormick.com>
Co-authored-by: Samuel Gerber <samuel.gerber@kitware.com>
Co-authored-by: Tom Birdsong <tom.birdsong@kitware.com>
Add Modules/ThirdParty/proxTV, a third-party module that builds the
proxTV library via FetchContent from the InsightSoftwareConsortium
fork (pinned in proxTVGitTag.cmake). The module is EXCLUDE_FROM_DEFAULT
and only configured when a dependent module is enabled.
ITK_USE_SYSTEM_proxTV selects an external build.

Uses FetchContent_MakeAvailable so proxTV is built as a proper CMake
subdirectory target, giving correct export and install behavior.
proxTV builds with the Eigen backend against ITK's in-tree Eigen3,
under ITK's C++17 standard.

Rewrite the ingested TotalVariation CMakeLists.txt for the in-tree
build (itk_module_impl, no project()/ITKModuleExternal), add ITKproxTV
to its dependencies, and use a literal module description.
TotalVariation is now ingested in-tree at Modules/Filtering/TotalVariation;
drop the configure-time fetch declaration.
Build the in-tree TotalVariation module (EXCLUDE_FROM_DEFAULT) in CI.
OutputImageType aliased TInputImage, so the second template parameter
(and the Superclass alias) silently ignored the requested output type;
use TOutputImage. Cast the image size to int explicitly for the proxTV
PD_TV dims argument, and fix the closing header-guard comment.
@hjmjohnson
hjmjohnson force-pushed the ingest-TotalVariation branch from cf0be3d to f185688 Compare June 5, 2026 15:53
@hjmjohnson

Copy link
Copy Markdown
Member Author

@blowekamp Rebased onto latest upstream/main (now at 3274f3029b). As you noted, the un-rooted ingest merge + orphan module branch survived a merge-preserving rebase intact — the ingest merge 5541af6c keeps its original second parent bf442db6a3 (the proxTV/TotalVariation module tip), only the first parent moved to current main, and the four follow-on fixups replayed on top. The only conflict was the one-line pyproject.toml CI-module list (your TwoProjectionRegistration landed in the same slot); both are now enabled alphabetically. pre-commit run --all-files is clean. CI re-running on f185688419.

Verification
merges since upstream/main: 11 (1 ingest + 10 preserved upstream module merges)
ingest merge parents: main(3274f3029b) + module-tip(bf442db6a3, unchanged)
fixups (first-parent): proxTV third-party / remove remote decl / enable in CI / harden ProxTVImageFilter
net diff vs main: TotalVariation + proxTV module trees, pyproject.toml, removed Modules/Remote/TotalVariation.remote.cmake

The fetched proxTV uses find_package(Eigen3) only as a fallback when
proxTV_Eigen_LIBRARIES is unset; the in-tree module left it empty (the
header-only ITKEigen3_LIBRARIES), so the fallback ran and failed during
the ITK build. Pass the ITK::ITKEigen3Module interface target instead,
and add the itkeigen include directory so proxTV's <Eigen/...> includes
resolve against ITK's prefixed Eigen headers.
@hjmjohnson

Copy link
Copy Markdown
Member Author

@blowekamp Fixed the proxTV/Eigen3 in-tree wiring (25745550e) along the modern interface-module lines you set up in the fork — verified with a full local build of the TotalVariation module and all 3 of its tests passing (incl. itkProxTVImageFilterTest).

Root cause + fix

Your fork only calls find_package(Eigen3) as a fallback (elseif(NOT proxTV_Eigen_LIBRARIES)). The in-tree module was setting proxTV_Eigen_LIBRARIES ${ITKEigen3_LIBRARIES}, which is empty for the header-only internal Eigen3, so the fallback ran and died during the ITK build. Fix:

  • set(proxTV_Eigen_LIBRARIES ITK::ITKEigen3Module) — the interface target, so the fork skips find_package entirely.
  • target_include_directories(proxTV SYSTEM PRIVATE "${ITKEigen3_SOURCE_DIR}/src/itkeigen") — proxTV sources use #include <Eigen/...>, but ITK vendors Eigen under the itkeigen/ prefix, so the directory holding Eigen/ directly is added (PRIVATE: only lapackFunctionsWrap.cpp includes Eigen; no public header does).

Local validation (Ninja, internal Eigen3): configure clean → proxTV builds → TotalVariationTestDriver builds → ctest -R TotalVariation 3/3 passed.

@hjmjohnson

Copy link
Copy Markdown
Member Author

@blowekamp Some additional changes needed to get the builds to pass.

@hjmjohnson
hjmjohnson requested a review from blowekamp June 5, 2026 20:33

@blowekamp blowekamp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, this is fine it works.

I should have uninstalled the system Eigen locally to see this error mode.

The added ITK::ITKEigen3Module rep doesn't really add libraries or include or anything. And adding that manual path should really happen. I guess ideally there should be an "ITK::Eigen3"interface library that is the raw unguarded headers... or provTV should be updated to use the ITKEIGEN(x) macro.

But this is fine for now.

@hjmjohnson
hjmjohnson merged commit 3609b55 into InsightSoftwareConsortium:main Jun 6, 2026
18 of 19 checks passed
hjmjohnson added a commit to InsightSoftwareConsortium/ITKTotalVariation that referenced this pull request Jun 6, 2026
The TotalVariation module is now maintained in ITK main at
Modules/Filtering/TotalVariation (ingested via
InsightSoftwareConsortium/ITK#6381; the bundled proxTV dependency now
lives at Modules/ThirdParty/proxTV). Delete whitelisted sources, rename
README.rst to info.rst, and promote the migration notice to README.md so
the GitHub landing page shows archived status.
@hjmjohnson
hjmjohnson deleted the ingest-TotalVariation branch August 22, 2026 15:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Filtering Issues affecting the Filtering module area:Python wrapping Python bindings for a class area:Remotes Issues affecting the Remote module area:ThirdParty Issues affecting the ThirdParty module type:Enhancement Improvement of existing methods or implementation type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct

Projects

None yet

Development

Successfully merging this pull request may close these issues.