Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
880a882
Add GPU hamming improvements
felixpetschko Jul 1, 2026
3d40391
Change cutoff guard to <=125
felixpetschko Jul 1, 2026
d053bee
Add parameter guards for block size parameters
felixpetschko Jul 1, 2026
5200588
Add tests for different block size parameters
felixpetschko Jul 1, 2026
5b89ab2
Adapt GPUHammingDistanceCalculator doc string
felixpetschko Jul 1, 2026
7718e2c
Change sequence length array data type to int32
felixpetschko Jul 1, 2026
9d3252d
Avoid unnecessary dtype copies in GPU hamming transfers
felixpetschko Jul 1, 2026
80ee4c5
Replace asserts with exceptions
felixpetschko Jul 1, 2026
dada164
Transform guarding assert into error
felixpetschko Jul 7, 2026
6887bf3
Remove benchmarking option
felixpetschko Jul 7, 2026
86391e7
Skip calculation of blocks under diagonal
felixpetschko Jul 7, 2026
1ab2ffb
Update block skipping
felixpetschko Jul 7, 2026
701b860
Update block skipping to work with tqdm
felixpetschko Jul 7, 2026
c82add0
Remove logic to only compute upper triangular half for symmetric resu…
felixpetschko Jul 7, 2026
4884d50
Extend test cases to cover block skipping
felixpetschko Jul 7, 2026
efa31eb
Update changelog
felixpetschko Jul 7, 2026
a2bba48
Add GPU hamming integration test
felixpetschko Jul 7, 2026
940d3a0
Prevent out-of-bounds writes in GPU Hamming kernel
felixpetschko Jul 22, 2026
d5eef6e
Handle excess blocks in GPU Hamming calculator
felixpetschko Jul 22, 2026
0530c30
Support joblib blocks in GPU Hamming calculator
felixpetschko Jul 22, 2026
4c9f30e
Balance symmetric distance blocks and use fixed GPU tile sizes
felixpetschko Jul 23, 2026
6d22b8b
Improve GPU Hamming block sizing and buffer handling
felixpetschko Jul 23, 2026
c5938ab
Change parameter names
felixpetschko Jul 23, 2026
7f4b593
Log tile retry info
felixpetschko Jul 23, 2026
14e0bcf
Test maximum GPU Hamming cutoff
felixpetschko Jul 23, 2026
9fea04d
Extend large dataset tutorial
felixpetschko Jul 23, 2026
717974b
Reformat
felixpetschko Jul 23, 2026
7feb2b4
Adapt input array memory management
felixpetschko Jul 26, 2026
3b62993
Avoid redundant host allocations for GPU Hamming result buffers
felixpetschko Sep 11, 2026
ef7b651
Reuse host indptr in GPU Hamming CSR assembly
felixpetschko Sep 11, 2026
c85a43a
Avoid retaining redundant GPU Hamming input buffers
felixpetschko Sep 11, 2026
6483650
Avoid temporary sequence tuples in GPU Hamming
felixpetschko Sep 11, 2026
32d097c
Prevent theoretical overflow in GPU Hamming combined entry count
felixpetschko Sep 11, 2026
feffe20
Use int32 for empty GPU Hamming result tiles
felixpetschko Sep 11, 2026
42468c2
Precompute sequence length ranges for GPU Hamming tiles
felixpetschko Sep 11, 2026
8243bb5
Merge upstream/main into performance/gpu-hamming-improvements
felixpetschko Sep 11, 2026
1ad0221
Deprecate legacy GPU Hamming parameters
felixpetschko Sep 11, 2026
a211e40
Clarify multi-GPU configuration and link to advanced usage docs
felixpetschko Sep 11, 2026
772055e
Add TCRdist GPU implementation
felixpetschko Sep 14, 2026
3285c8f
Strengthen GPU TCRdist buffer retry and length boundary tests
felixpetschko Sep 14, 2026
329408c
Add changelog entry for GPU TCRdist
felixpetschko Sep 14, 2026
6a595f9
Document GPU TCRdist support in the large datasets tutorial
felixpetschko Sep 14, 2026
f64582d
Increase default TCRdist gap penalty to 12 and warn about the change
felixpetschko Sep 28, 2026
e1c5a47
Exclude fully trimmed sequences from TCRdist comparisons
felixpetschko Sep 28, 2026
24efc9f
Respect custom trimming when positioning fixed TCRdist gaps
felixpetschko Sep 28, 2026
b24fab9
Increase default TCRdist cutoff to 24 and consolidate default-change …
felixpetschko Sep 28, 2026
802f5ac
Add TODO for removing temporary TCRdist default-change warnings
felixpetschko Sep 28, 2026
e20a3c6
Document original fixed gap-position formula and custom trimming adap…
felixpetschko Sep 28, 2026
80df9f0
Merge remote-tracking branch 'upstream/main' into feature/gpu-tcrdist
felixpetschko Sep 30, 2026
b0b85f4
Merge branch 'fix/tcrdist-parameters-trimming' into feature/gpu-tcrdist
felixpetschko Sep 30, 2026
e61f789
Update default cutoff and gap_penalty
felixpetschko Sep 30, 2026
4b54cb1
Avoid comparison of fully trimmed sequences
felixpetschko Sep 30, 2026
7ea6d99
Respect custom trimming when calculating fixed GPU TCRdist gap position
felixpetschko Sep 30, 2026
93d5a57
Update GPU TCRdist documentation for new defaults and trimming behavior
felixpetschko Sep 30, 2026
00e62ab
Update GPU TCRdist tests for new defaults and trimming behavior
felixpetschko Sep 30, 2026
2e79835
Precompute weighted amino acid distance matrix
felixpetschko Sep 30, 2026
b17b75b
Exclude fully trimmed columns from GPU TCRdist length bounds
felixpetschko Sep 30, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,31 @@ and this project adheres to [Semantic Versioning][].

## Unreleased

### Features

- Add `metric="gpu_tcrdist"` for GPU-accelerated TCRdist. Currently only `fixed_gappos=True` is supported.

### Changed

- Increase the default TCRdist `cutoff` from 20 to 24, allowing two gap positions with the new
default gap penalty of 12. Omitting `cutoff` emits a `UserWarning`, combined with the gap-penalty
warning when both defaults are used. Set `cutoff=24` explicitly to acknowledge the new default,
or `cutoff=20` to retain the previous cutoff. To retain both previous parameter values, also set
`gap_penalty=4`. The increased cutoff can add sequence matches and change clonotype clusters.

- Make the fixed TCRdist gap position depend on `ntrim` and `ctrim` so it stays within the
remaining region. Default trimming retains the previous behavior; custom trimming can change
distances. The variable gap-position search (`fixed_gappos=False`) is unchanged.

- Exclude TCRdist comparisons involving sequences with no positions remaining after trimming,
including self-comparisons. A warning reports the number of affected input sequences; matrix
shape and sequence order are preserved.

- Increase the default TCRdist `gap_penalty` from 4 to 12 to match the authors' later implementation.
This changes distances between sequences of different lengths. Omitting `gap_penalty` now emits a
`UserWarning`; explicitly set `gap_penalty=12` to acknowledge the new default, or `gap_penalty=4`
to retain the previous behavior.

### Fixes

- Avoid deprecated AnnData `obsm_keys()` calls when plotting clonotype networks or exporting them to igraph.
Expand Down
1 change: 1 addition & 0 deletions docs/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -311,4 +311,5 @@ distance metrics
ir_dist.metrics.AlignmentDistanceCalculator
ir_dist.metrics.FastAlignmentDistanceCalculator
ir_dist.metrics.TCRdistDistanceCalculator
ir_dist.metrics.GPUTCRdistDistanceCalculator
ir_dist.metrics.NeedlemanWunschDistanceCalculator
5 changes: 3 additions & 2 deletions docs/tutorials/large-datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,10 @@ with joblib.parallel_config(backend="dask", n_jobs=200, verbose=10):
)
```

## Using GPU acceleration for hamming distance
## Using GPU acceleration for Hamming and TCRdist

The Hamming distance metric supports GPU acceleration via [cupy](https://cupy.dev/).
The Hamming and TCRdist distance metrics support GPU acceleration via [cupy](https://cupy.dev/).
The examples below use `gpu_hamming`; `gpu_tcrdist` supports the same GPU tile parameters.

First, install the optional `cupy` dependency:

Expand Down
8 changes: 7 additions & 1 deletion src/scirpy/ir_dist/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ def IrNeighbors(*args, **kwargs):
"gpu_hamming",
"normalized_hamming",
"tcrdist",
"gpu_tcrdist",
"needleman_wunsch",
]
| metrics.DistanceCalculator
Expand All @@ -56,6 +57,9 @@ def IrNeighbors(*args, **kwargs):
Uses the BLOSUM62 substitution matrix by default. TCRBLOSUM alpha/beta substitution matrices
(:cite:`TCRBLOSUM`) can be selected with `base_matrix="tcrblosum"`.
See :class:`~scirpy.ir_dist.metrics.TCRdistDistanceCalculator`.
* `gpu_tcrdist` -- TCRdist calculated with a GPU, with support for BLOSUM62 and TCRBLOSUM matrices.
Currently only supports `fixed_gappos=True`.
See :class:`~scirpy.ir_dist.metrics.GPUTCRdistDistanceCalculator`.
* `needleman_wunsch` -- Distance based on linear-gap Needleman-Wunsch global alignment.
Uses the BLOSUM62 substitution matrix.
This option is incompatible with nucleotide sequences.
Expand Down Expand Up @@ -84,7 +88,7 @@ def IrNeighbors(*args, **kwargs):
matrix. A sensible cutoff depends on the distance metric, you can find
information in the corresponding docs. If set to `None`, the cutoff
will be `10` for the `alignment`, `fastalignment`, and `needleman_wunsch` metric,
and `2` for `levenshtein` and `hamming`.
`24` for `tcrdist` and `gpu_tcrdist`, and `2` for `levenshtein` and `hamming`.
For the identity metric, the cutoff is ignored and always set to `0`.
"""

Expand Down Expand Up @@ -127,6 +131,8 @@ def _get_distance_calculator(
dist_calc = metrics.GPUHammingDistanceCalculator(**kwargs)
elif metric == "tcrdist":
dist_calc = metrics.TCRdistDistanceCalculator(n_jobs=n_jobs, chain_type=chain_type, **kwargs)
elif metric == "gpu_tcrdist":
dist_calc = metrics.GPUTCRdistDistanceCalculator(chain_type=chain_type, **kwargs)
elif metric == "needleman_wunsch":
dist_calc = metrics.NeedlemanWunschDistanceCalculator(n_jobs=n_jobs, **kwargs)
else:
Expand Down
2 changes: 1 addition & 1 deletion src/scirpy/ir_dist/_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ def lookup(

distance_matrix = self.distance_matrices[distance_matrix_name]

if np.max(distance_matrix.data) > np.iinfo(np.uint8).max:
if np.max(distance_matrix.data, initial=0) > np.iinfo(np.uint8).max:
raise OverflowError(
"The data values in the distance scipy.sparse.csr_matrix exceed the maximum value for uint8 (255)"
)
Expand Down
Loading
Loading