diff --git a/README.md b/README.md index 4218a8a79..f1af10192 100644 --- a/README.md +++ b/README.md @@ -235,6 +235,10 @@ This tutorial demonstrates how to construct a training workflow of [HoVerNet](ht ##### [Nuclei Classification](./pathology/nuclick#nuclei-classification-model) The notebook demonstrates examples of training and inference pipelines with interactive annotation for pathology, NuClick is used for delineating nuclei, cells and a squiggle for outlining glands. +#### **Multimodal** +##### [Multimodal Early-Fusion Network](./multimodal/nakaseke_multimodal_early_fusion) +An end-to-end tutorial fusing a 2D radiograph stream (MONAI `DenseNet121`) with a low-dimensional clinical tabular stream via dictionary-based transforms and `torch.cat`, using a fully synthetic, locally generated dataset. + #### **Acceleration** ##### [fast_model_training_guide](./acceleration/fast_model_training_guide.md) The document introduces details of how to profile the training pipeline, how to analyze the dataset and select suitable algorithms, and how to optimize GPU utilization in single GPU, multi-GPUs or even multi-nodes. diff --git a/multimodal/nakaseke_multimodal_early_fusion/README.md b/multimodal/nakaseke_multimodal_early_fusion/README.md new file mode 100644 index 000000000..a5832c6ba --- /dev/null +++ b/multimodal/nakaseke_multimodal_early_fusion/README.md @@ -0,0 +1,54 @@ +# Multimodal Early-Fusion Network: Radiographs + Clinical Tabular Data + +[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Project-MONAI/tutorials/blob/main/multimodal/nakaseke_multimodal_early_fusion/multimodal_early_fusion_tutorial.ipynb) + +This tutorial demonstrates an **early-fusion** architecture that combines a 2D medical image stream +with a low-dimensional clinical tabular stream in a single MONAI dictionary-based pipeline, using +[`multimodal_early_fusion_tutorial.ipynb`](./multimodal_early_fusion_tutorial.ipynb). + +## Motivation + +Most MONAI tutorials focus on a single imaging modality. In real deployments, especially in +resource-constrained clinics, an image is rarely read in isolation -- a clinician also has vitals, +labs, or a short structured history on hand, and a model that ignores that context is throwing away +signal it doesn't have to. This tutorial shows the minimum end-to-end MONAI/PyTorch pattern for +combining the two without hand-rolling a custom `Dataset`. + +The clinical feature schema (age, BMI, salivary pH, systolic blood pressure) and the binary screening +task are modeled after a hypertension-screening workflow explored at Nakaseke Hospital, Uganda. The +underlying patient data is confidential and is **not** included in or downloaded by this tutorial. + +## Dataset + +**Fully synthetic, generated locally, no download required.** The notebook's +`simulate_nakaseke_multimodal_dataset()` function creates, on the fly: + +- one synthetic 2D radiograph per patient, saved as a NIfTI (`.nii.gz`) file with `nibabel` +- a 4-dimensional tabular vector `[age, bmi, salivary_ph, systolic_bp]` +- a binary label + +Both modalities are generated from a shared hidden "risk factor" per patient, so neither the image +nor the tabular vector is fully predictive of the label on its own -- this is what motivates fusing +them. There is no claim that the notebook's results reflect real-world diagnostic performance; the +synthetic cohort exists only to exercise the pipeline end-to-end without any real or downloadable +data. + +## What the notebook covers + +1. Reproducible setup with `monai.utils.set_determinism`. +2. Synthetic multimodal data generation and a MONAI-style data manifest + (`{"image": ..., "nakaseke_tabular": ..., "label": ...}`). +3. A dictionary-based `Compose` pipeline where image-only transforms (`LoadImaged`, + `EnsureChannelFirstd`, `ScaleIntensityRanged`) touch only the `"image"` key, while the tabular and + label keys are cast to tensors with `EnsureTyped` and otherwise left untouched. +4. `ResilientMultimodalClassifier`: a MONAI `DenseNet121` visual stream (512-d embedding) fused via + `torch.cat` with a small tabular projection stream (16-d embedding) into a 528-d representation, + followed by a dropout-regularized classification head. +5. A short training loop (`max_epochs`, `val_interval`) showing validation accuracy improve as the + model learns to use both streams. + +## Requirements + +Everything needed is installed by the notebook's own `Setup environment` cell +(`monai-weekly[nibabel, tqdm]`, `matplotlib`). No GPU is required -- the default image size (64x64) +and cohort size (200 patients) are chosen to train in well under a minute on CPU. diff --git a/multimodal/nakaseke_multimodal_early_fusion/multimodal_early_fusion_tutorial.ipynb b/multimodal/nakaseke_multimodal_early_fusion/multimodal_early_fusion_tutorial.ipynb new file mode 100644 index 000000000..95b0bd1db --- /dev/null +++ b/multimodal/nakaseke_multimodal_early_fusion/multimodal_early_fusion_tutorial.ipynb @@ -0,0 +1,544 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Copyright (c) MONAI Consortium \n", + "Licensed under the Apache License, Version 2.0 (the \"License\"); \n", + "you may not use this file except in compliance with the License. \n", + "You may obtain a copy of the License at \n", + "    http://www.apache.org/licenses/LICENSE-2.0 \n", + "Unless required by applicable law or agreed to in writing, software \n", + "distributed under the License is distributed on an \"AS IS\" BASIS, \n", + "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. \n", + "See the License for the specific language governing permissions and \n", + "limitations under the License.\n", + "\n", + "# Multimodal Early-Fusion Network: Radiographs + Clinical Tabular Data\n", + "\n", + "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/Project-MONAI/tutorials/blob/main/multimodal/nakaseke_multimodal_early_fusion/multimodal_early_fusion_tutorial.ipynb)\n", + "\n", + "This tutorial builds an end-to-end **early-fusion** classifier that combines a 2D medical image stream with a low-dimensional clinical tabular stream, using only MONAI's dictionary-based transform and dataset APIs plus a small PyTorch fusion network.\n", + "\n", + "The clinical schema (age, BMI, salivary pH, systolic blood pressure) and the binary screening task are modeled after a real hypertension-screening workflow at Nakaseke Hospital, Uganda. Because that patient data is confidential and cannot be published, **this notebook generates a fully synthetic cohort locally** -- no downloads, no external services, no real patient data -- while keeping the same feature schema, tensor shapes, and modeling problem, so the pipeline is a drop-in template for a real (IRB-approved, de-identified) dataset with matching keys.\n", + "\n", + "The synthetic generator ties both modalities to a shared hidden \"risk factor\" per patient, so neither the image nor the tabular vector is fully predictive on its own -- this is what makes the fusion architecture worth demonstrating, rather than a stream that could win alone.\n", + "\n", + "**Why early fusion, and why it fits this problem specifically.** Fusion strategies are usually grouped into early (feature-level), late (decision-level), and joint/intermediate variants. Late fusion -- training an image classifier and a tabular classifier independently and averaging their predictions -- is the more common shortcut, but it caps the model's expressive power at the sum of two univariate opinions: it cannot learn that, say, a mildly ambiguous radiograph combined with a specific BMI/BP combination is jointly more informative than either reading alone. Early fusion, by concatenating learned embeddings from both streams before a shared classification head, lets the network learn exactly those cross-modal interactions. The cost is exactly what this notebook is designed to show how to pay safely: image and tabular data live on very different scales and shapes, and MONAI's dictionary transforms are what make combining them straightforward without writing a custom `Dataset`.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup environment" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "!python -c \"import monai\" || pip install -q \"monai-weekly[nibabel, tqdm]\"\n", + "!python -c \"import matplotlib\" || pip install -q matplotlib\n", + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup imports" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import shutil\n", + "import tempfile\n", + "from typing import Any\n", + "\n", + "import matplotlib.pyplot as plt\n", + "import nibabel as nib\n", + "import numpy as np\n", + "import torch\n", + "import torch.nn as nn\n", + "from monai.config import print_config\n", + "from monai.data import DataLoader, Dataset\n", + "from monai.metrics import ROCAUCMetric\n", + "from monai.networks.nets import DenseNet121\n", + "from monai.transforms import Compose, EnsureChannelFirstd, EnsureTyped, LoadImaged, ScaleIntensityRanged\n", + "from monai.utils import set_determinism\n", + "\n", + "print_config()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Reproducibility\n", + "\n", + "MONAI's `set_determinism` seeds Python's `random`, NumPy, and PyTorch (including CUDA) in one call, so the synthetic cohort, model initialization, and training loop below are reproducible run to run." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "set_determinism(seed=42)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup data directory\n", + "\n", + "You can specify a directory with the `MONAI_DATA_DIRECTORY` environment variable. \n", + "This allows you to save results and reuse the generated files. \n", + "If not specified, a temporary directory is used and removed at the end of the notebook." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "directory = os.environ.get(\"MONAI_DATA_DIRECTORY\")\n", + "if directory is not None:\n", + " os.makedirs(directory, exist_ok=True)\n", + "root_dir = tempfile.mkdtemp() if directory is None else directory\n", + "print(root_dir)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The clinical data generator (Nakaseke-inspired, fully synthetic)\n", + "\n", + "`simulate_nakaseke_multimodal_dataset` creates, entirely on disk locally:\n", + "- one 2D synthetic radiograph slice per patient, saved as a standard NIfTI (`.nii.gz`) file with `nibabel`\n", + "- a 4-dimensional tabular vector `[age, bmi, salivary_ph, systolic_bp]`\n", + "- a binary screening label\n", + "\n", + "and returns a MONAI-style data manifest: a list of dictionaries `{\"image\": path, \"nakaseke_tabular\": array, \"label\": int}`.\n", + "\n", + "**On the radiograph itself:** this is a *stylized synthetic density map*, not a rendering of real anatomy -- generated by low-pass filtering white noise into a smooth spatial field, then applying a radial vignette so density fades toward the edges (see the visualization below). The point is not to look like a diagnostic-quality X-ray; it is to give the CNN stream a spatially *smooth*, *bounded* structure whose average density is deliberately correlated with `risk_factor` -- unlike independent per-pixel noise, which a convolutional network could still exploit through its mean but which would not visually resemble anything a clinician would recognize as tissue." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "def _generate_smooth_density_field(rng: np.random.Generator, image_size: int, cutoff: float = 0.12) -> np.ndarray:\n", + " \"\"\"Low-pass filter white noise into a smooth, blob-like spatial field.\n", + "\n", + " Real tissue density varies smoothly in space; independent per-pixel noise\n", + " does not, and looks like static rather than anatomy. An FFT low-pass\n", + " filter is the simplest way to get that smoothness using only NumPy, so\n", + " the notebook stays free of extra plotting/image dependencies.\n", + " \"\"\"\n", + " noise = rng.normal(0.0, 1.0, size=(image_size, image_size))\n", + " freqs = np.fft.fftfreq(image_size)\n", + " freq_x, freq_y = np.meshgrid(freqs, freqs)\n", + " radial_frequency = np.sqrt(freq_x**2 + freq_y**2)\n", + " low_pass_mask = (radial_frequency < cutoff).astype(np.float32)\n", + " smoothed = np.fft.ifft2(np.fft.fft2(noise) * low_pass_mask).real\n", + " return smoothed / (smoothed.std() + 1e-6)\n", + "\n", + "\n", + "def simulate_nakaseke_multimodal_dataset(\n", + " root_dir: str,\n", + " num_patients: int = 200,\n", + " image_size: int = 64,\n", + " seed: int = 42,\n", + ") -> list[dict[str, Any]]:\n", + " \"\"\"Generate a synthetic multimodal cohort mimicking the Nakaseke Hospital\n", + " hypertension-screening schema (2D radiograph + 4 clinical features).\n", + "\n", + " No real patient data is used or required. A shared latent ``risk_factor``\n", + " per patient drives both the systolic blood pressure and the mean\n", + " radiograph density, so that neither modality alone is fully predictive\n", + " of the label.\n", + " \"\"\"\n", + " image_dir = os.path.join(root_dir, \"nakaseke_synthetic_images\")\n", + " os.makedirs(image_dir, exist_ok=True)\n", + "\n", + " rng = np.random.default_rng(seed)\n", + " affine = np.eye(4)\n", + " data_manifest: list[dict[str, Any]] = []\n", + "\n", + " # Radial vignette: fades density toward the edges so each synthetic scan\n", + " # reads as one bounded structure rather than texture filling the frame.\n", + " # It is identical for every patient, so it is computed once, outside the loop.\n", + " yy, xx = np.mgrid[0:image_size, 0:image_size]\n", + " center = (image_size - 1) / 2\n", + " radius_from_center = np.sqrt((xx - center) ** 2 + (yy - center) ** 2)\n", + " vignette = np.clip(1.0 - (radius_from_center / radius_from_center.max()) ** 1.5, 0.0, 1.0)\n", + "\n", + " for patient_idx in range(num_patients):\n", + " risk_factor = rng.normal(loc=0.0, scale=1.0)\n", + "\n", + " age = float(np.clip(rng.normal(45, 15), 18, 90))\n", + " bmi = float(np.clip(rng.normal(24, 5), 15, 45))\n", + " salivary_ph = float(np.clip(rng.normal(6.8, 0.4), 5.5, 8.0))\n", + " systolic_bp = float(np.clip(125 + 15 * risk_factor + rng.normal(0, 8), 90, 200))\n", + " tabular_features = np.array([age, bmi, salivary_ph, systolic_bp], dtype=np.float32)\n", + "\n", + " base_intensity = 90.0 + 25.0 * risk_factor\n", + " density_field = _generate_smooth_density_field(rng, image_size)\n", + " structure = 55.0 * density_field * vignette\n", + " fine_grain = rng.normal(0, 6, size=(image_size, image_size))\n", + " radiograph = np.clip(base_intensity + structure + fine_grain, 0, 255).astype(np.float32)\n", + "\n", + " label_logit = risk_factor + rng.normal(0, 0.6)\n", + " label = int(label_logit > 0)\n", + "\n", + " scan_path = os.path.join(image_dir, f\"patient_{patient_idx:04d}.nii.gz\")\n", + " nib.save(nib.Nifti1Image(radiograph, affine), scan_path)\n", + "\n", + " data_manifest.append(\n", + " {\n", + " \"image\": scan_path,\n", + " \"nakaseke_tabular\": tabular_features,\n", + " \"label\": label,\n", + " }\n", + " )\n", + "\n", + " return data_manifest\n", + "\n", + "\n", + "data_manifest = simulate_nakaseke_multimodal_dataset(root_dir, num_patients=200, image_size=64)\n", + "train_files, val_files = data_manifest[:160], data_manifest[160:]\n", + "\n", + "print(\n", + " f\"Generated {len(data_manifest)} synthetic patient records \"\n", + " f\"({len(train_files)} train / {len(val_files)} validation).\"\n", + ")\n", + "print(\"Example record keys:\", list(data_manifest[0].keys()))\n", + "print(\"Example tabular vector [age, bmi, salivary_ph, systolic_bp]:\", data_manifest[0][\"nakaseke_tabular\"])\n", + "print(\"Example label:\", data_manifest[0][\"label\"])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Advanced dictionary transforms: preserving the tabular stream\n", + "\n", + "The `Compose` pipeline below only ever targets the `\"image\"` key for image-specific processing\n", + "(`LoadImaged`, `EnsureChannelFirstd`, `ScaleIntensityRanged` to normalize scanner variation that is\n", + "common across rural-clinic radiograph equipment). The `\"nakaseke_tabular\"` and `\"label\"` entries never\n", + "pass through any image transform -- they are only explicitly cast to PyTorch tensors with `EnsureTyped`,\n", + "which is what lets a single dictionary-based pipeline carry heterogeneous, non-image data safely\n", + "alongside imaging data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "multimodal_transforms = Compose(\n", + " [\n", + " LoadImaged(keys=\"image\", image_only=True),\n", + " EnsureChannelFirstd(keys=\"image\"),\n", + " ScaleIntensityRanged(keys=\"image\", a_min=0, a_max=255, b_min=0.0, b_max=1.0, clip=True),\n", + " EnsureTyped(keys=[\"nakaseke_tabular\", \"label\"]),\n", + " ]\n", + ")\n", + "\n", + "train_ds = Dataset(data=train_files, transform=multimodal_transforms)\n", + "val_ds = Dataset(data=val_files, transform=multimodal_transforms)\n", + "\n", + "train_loader = DataLoader(train_ds, batch_size=8, shuffle=True, num_workers=0)\n", + "val_loader = DataLoader(val_ds, batch_size=8, shuffle=False, num_workers=0)\n", + "\n", + "sanity_batch = next(iter(train_loader))\n", + "print(\"Sanity check on one batch:\")\n", + "print(\" image shape: \", sanity_batch[\"image\"].shape)\n", + "print(\" nakaseke_tabular shape: \", sanity_batch[\"nakaseke_tabular\"].shape)\n", + "print(\" label shape: \", sanity_batch[\"label\"].shape)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Visualize one synthetic sample\n", + "\n", + "Expect a soft, blobby grayscale pattern that fades out toward the edges -- not a sharp anatomical image, and not flat noise either. If it instead looks like pure salt-and-pepper static, `_generate_smooth_density_field`'s FFT low-pass step did not run; if it looks like a hard geometric grid, the vignette exponent is too aggressive for the chosen `image_size`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "sample = val_ds[0]\n", + "plt.imshow(sample[\"image\"][0], cmap=\"gray\")\n", + "plt.title(f\"Synthetic radiograph | label={int(sample['label'])}\")\n", + "plt.axis(\"off\")\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The multimodal early-fusion architecture\n", + "\n", + "`ResilientMultimodalClassifier` has two independent streams and a fusion junction:\n", + "\n", + "- **Stream 1 (visual):** a MONAI `DenseNet121` (`spatial_dims=2, in_channels=1, out_channels=512`) that\n", + " turns each radiograph into a 512-dimensional embedding.\n", + "- **Stream 2 (tabular context):** a small feed-forward projection network that compresses the\n", + " 4-dimensional Nakaseke tabular vector into a 16-dimensional embedding.\n", + "- **Fusion junction:** the two embeddings are concatenated with `torch.cat` into a single\n", + " 528-dimensional (512 + 16) fused representation.\n", + "- **Classification head:** a dropout-regularized linear head maps the fused representation to the\n", + " final diagnostic logit.\n", + "\n", + "**Design intuition worth calling out explicitly:**\n", + "\n", + "- *Why the 512 vs. 16 embedding asymmetry?* An image carries far more raw entropy than a 4-value\n", + " vector, so collapsing both streams to the same width would either waste capacity on the tabular side\n", + " or force the image into an unnecessarily narrow bottleneck. 16 dimensions is enough for a linear\n", + " projection of 4 clinical features to be useful in the fused vector without dominating it. This is a\n", + " practical default, not a tuned optimum -- a natural extension is a learned gating or attention layer\n", + " that lets the network decide the relative weight of each stream per patient instead of fixing it\n", + " through embedding width alone.\n", + "- *Why Dropout in the head, specifically?* Real single-site clinical tabular cohorts are usually small\n", + " (tens to a few hundred patients), while `DenseNet121` alone carries roughly 7-8 million parameters.\n", + " Dropout on the fused representation is a cheap, standard regularizer for exactly this\n", + " small-data/large-model regime; for a production deployment on a real cohort this size, pairing it\n", + " with weight decay and early stopping on a held-out validation loss is advisable.\n", + "- *Why not just concatenate the raw 4 tabular values instead of projecting them?* Because the raw\n", + " values live on very different scales (age in years vs. salivary pH in single digits), and\n", + " concatenating unprojected features next to a 512-d embedding would let the image stream dominate the\n", + " fused gradient by sheer dimensionality. A small learned projection gives the optimizer a\n", + " same-order-of-magnitude representation to work with on both sides of `torch.cat`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "class ResilientMultimodalClassifier(nn.Module):\n", + " \"\"\"Early-fusion network combining a 2D radiograph stream with a tabular clinical stream.\"\"\"\n", + "\n", + " def __init__(\n", + " self,\n", + " tabular_in_features: int = 4,\n", + " tabular_embed_dim: int = 16,\n", + " image_embed_dim: int = 512,\n", + " num_classes: int = 1,\n", + " dropout: float = 0.3,\n", + " ) -> None:\n", + " super().__init__()\n", + "\n", + " self.image_stream = DenseNet121(\n", + " spatial_dims=2,\n", + " in_channels=1,\n", + " out_channels=image_embed_dim,\n", + " )\n", + "\n", + " self.tabular_stream = nn.Sequential(\n", + " nn.Linear(tabular_in_features, 32),\n", + " nn.ReLU(),\n", + " nn.Linear(32, tabular_embed_dim),\n", + " nn.ReLU(),\n", + " )\n", + "\n", + " fused_dim = image_embed_dim + tabular_embed_dim\n", + " self.classification_head = nn.Sequential(\n", + " nn.Dropout(dropout),\n", + " nn.Linear(fused_dim, 64),\n", + " nn.ReLU(),\n", + " nn.Dropout(dropout),\n", + " nn.Linear(64, num_classes),\n", + " )\n", + "\n", + " def forward(self, image: torch.Tensor, tabular: torch.Tensor) -> torch.Tensor:\n", + " image_embedding = self.image_stream(image)\n", + " tabular_embedding = self.tabular_stream(tabular)\n", + " fused = torch.cat([image_embedding, tabular_embedding], dim=1)\n", + " return self.classification_head(fused)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Prove the fusion graph with a single mock forward pass" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n", + "model = ResilientMultimodalClassifier().to(device)\n", + "\n", + "mock_images = torch.randn(4, 1, 64, 64, device=device)\n", + "mock_tabular = torch.randn(4, 4, device=device)\n", + "mock_logits = model(mock_images, mock_tabular)\n", + "\n", + "print(\"Mock fused output shape:\", mock_logits.shape)\n", + "assert mock_logits.shape == (4, 1), \"Fusion graph produced an unexpected output shape.\"\n", + "print(\"Fusion graph OK: image (512-d) + tabular (16-d) -> 528-d fused vector -> 1 logit.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Train the fusion model\n", + "\n", + "A short training loop over the synthetic cohort. `max_epochs` and `val_interval` follow the MONAI\n", + "tutorial convention so the automated notebook-execution tests can safely reduce them for CI." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "max_epochs = 5\n", + "val_interval = 1\n", + "learning_rate = 1e-3\n", + "\n", + "loss_function = nn.BCEWithLogitsLoss()\n", + "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\n", + "\n", + "for epoch in range(max_epochs):\n", + " model.train()\n", + " epoch_loss = 0.0\n", + " for batch in train_loader:\n", + " images = batch[\"image\"].to(device)\n", + " tabular = batch[\"nakaseke_tabular\"].to(device)\n", + " labels = batch[\"label\"].to(device).float().unsqueeze(1)\n", + "\n", + " optimizer.zero_grad()\n", + " logits = model(images, tabular)\n", + " loss = loss_function(logits, labels)\n", + " loss.backward()\n", + " optimizer.step()\n", + " epoch_loss += loss.item()\n", + "\n", + " epoch_loss /= len(train_loader)\n", + " print(f\"epoch {epoch + 1}/{max_epochs} average training loss: {epoch_loss:.4f}\")\n", + "\n", + " if (epoch + 1) % val_interval == 0:\n", + " model.eval()\n", + " correct, total = 0, 0\n", + " with torch.no_grad():\n", + " for batch in val_loader:\n", + " images = batch[\"image\"].to(device)\n", + " tabular = batch[\"nakaseke_tabular\"].to(device)\n", + " labels = batch[\"label\"].to(device).float().unsqueeze(1)\n", + "\n", + " logits = model(images, tabular)\n", + " predictions = (torch.sigmoid(logits) > 0.5).float()\n", + " correct += (predictions == labels).sum().item()\n", + " total += labels.numel()\n", + "\n", + " print(f\"epoch {epoch + 1}/{max_epochs} validation accuracy: {correct / total:.4f}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Report a threshold-independent clinical metric\n", + "\n", + "Accuracy depends on the 0.5 decision threshold and is a poor summary for screening tasks with any\n", + "class imbalance -- it is easy to look good on accuracy by mostly predicting the majority class. The\n", + "ROC-AUC is threshold-independent and is the metric clinical ML work is generally expected to report,\n", + "so this notebook computes it once on the full validation set with MONAI's native `ROCAUCMetric`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "auc_metric = ROCAUCMetric()\n", + "model.eval()\n", + "with torch.no_grad():\n", + " for batch in val_loader:\n", + " images = batch[\"image\"].to(device)\n", + " tabular = batch[\"nakaseke_tabular\"].to(device)\n", + " labels = batch[\"label\"].to(device).float().unsqueeze(1)\n", + "\n", + " probabilities = torch.sigmoid(model(images, tabular))\n", + " auc_metric(y_pred=probabilities, y=labels)\n", + "\n", + "final_val_auc = auc_metric.aggregate()\n", + "auc_metric.reset()\n", + "print(f\"Final validation ROC-AUC: {final_val_auc:.4f}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Cleanup data directory\n", + "\n", + "Remove the temporary directory if one was used." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "if directory is None:\n", + " shutil.rmtree(root_dir)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.12" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}