The goal of this project is to provide an easily modifiable and understandable pipeline for recreating MitoNet, a model for mitochondria panoptic segmentation from this paper: https://doi.org/10.1016/j.cels.2022.12.006. Any mentions of 'the paper' in this code refer to this work. The training is divided into self-supervised pretraining and training on a labeled dataset.
Most of the code has been taken from the following repositories:
I contributed by fixing bugs, adapting the code to match the specific task at hand, adding benchmark scripts and adapting encoders that were taken from:
MitoNet uses SwAV to pretrain on 1.5M unlabeled electron microscopy mitochondria images, the code from this part required minor changes
Done on labeled electron microscopy mitochondria images, the code from this part required major changes
To recreate MitoNet:
- Switch to environment fulfilling SwAV requirements
- Fill out the shell script in the SwAV directory and run it
- Switch to environment fulfilling MitoNet training requirements
- Fill out the train_config.yaml file
- Fill out the train.sh script and run it
To benchmark:
- Switch to the same environment as MitoNet
- Fill out the benchmark.yaml file
- Run the following:
python benchmark.py -c benchmark.yamlDifferent encoders:
| Family | Variants |
|---|---|
| ResNet | resnet18, resnet34, resnet50, resnet101, resnet152, resnext50_32x4d, resnext101_32x8d, wide_resnet50_2, wide_resnet101_2 |
| Swin | swin_t, swin_s, swin_b, swin_l |
| SwinV2 | swinv2_t, swinv2_s, swinv2_b, swinv2_l |
| ConvNeXt | convnext_tiny, convnext_small, convnext_base, convnext_large |
- All encoders use the PanopticDeepLab architecture, changing the decoder_channels and aspp_channels parameters as well as crop sizes in the training transformations may be necessary when training with Swin encoders.
- Pretraining on different encoders than ResNet50 is not supported
Results were benchmarked with global set to True and thing_area set to 150 in benchmark.yaml
| encoder | pretrain | semantic_iou | f1@50 | f1@75 | AP@50 | AP@75 | PQ | MSA | #params | model |
|---|---|---|---|---|---|---|---|---|---|---|
| resnet50 | CEM1.5M | 0.771 | 0.757 | 0.669 | 0.609 | 0.502 | 0.662 | 0.455 | 25M | HuggingFace |
| resnet50 | ImageNet | 0.764 | 0.768 | 0.677 | 0.624 | 0.511 | 0.671 | 0.464 | 25M | HuggingFace |
| ConvNeXt Large | LVD-1689M (DINOv3) | 0.800 | 0.839 | 0.676 | 0.722 | 0.510 | 0.694 | 0.465 | 198M | HuggingFace |
| SwinV2-S | ImageNet-1k | 0.820 | 0.765 | 0.668 | 0.619 | 0.502 | 0.665 | 0.455 | 50M | HuggingFace |