Project page Documentation Changelog nicetoolbox@tue.mpg.de
🚀 We are releasing a new 0.3.1 version that adds new UniGaze gaze estimation, CrisperWhisper audio transcription, InsightFace face analysis, eye closure and blink detection, Py-Feat updated to a new version, audio evaluation metrics, a reworked installation, and many other improvements and fixes. Please check the changelog for more information.
NICE Toolbox is an easy-to-use framework for exploring nonverbal human communication. It aims to enable the investigation of observable signs that reflect the mental state and behaviors of the individual. Additionally, these visual nonverbal cues reveal the interpersonal dynamics between people in face-to-face conversations.
NICE combines existing computer vision detectors into a single, easy-to-use framework. Working from single- or multi-camera video data, it covers whole-body pose estimation, gaze tracking, movement dynamics (kinematics), gaze interaction monitoring (mutual gaze), physical proximity between dyads, emotion detection and more. For a full list, see the components overview and supported algorithms pages.
The toolbox also includes a visualizer module for interactively exploring outputs, an evaluation module that runs configurable metrics and a collection of connectors for importing/exporting data to third-party tools (e.g. for labelling in ELAN or napari-deeplabcut).
If you have additional questions or would like to collaborate on a project using NICE Toolbox, please reach out to us at nicetoolbox@tue.mpg.de.
For instructions on installing the toolbox on a Linux or Windows machine, please see the installation instructions page. For a quick start into the toolbox, we provide an example dataset and documentation to set it up on the getting started page. Further tutorials and documentation can be found on the tutorials and wiki pages. You can also access this documentation offline by downloading it as a PDF. Just use the ReadTheDocs pop-up menu located in the bottom right corner of the screen.
We acknowledge the following tools, methods, and frameworks: MMPose, HigherHRNet, ViTPose, DarkPose, RTMPose, MotionBERT, SAM 3D Body, SAM 2, MoGe, Detectron2, ETH-XGaze, UniGaze, SPIGA, InsightFace, face-alignment, WhisperX, CrisperWhisper, Py-Feat, and rerun.io.
Aleksandr Evgrashin, Carolin Schmitt, Timo Lübbing, Ashutosh Jha, Buket Naz Zeren, Sophie Bauer, Gökce Ergün, Senya Polikovsky.
All authors are with the Optics and Sensing Laboratory at Max Planck Institute for Intelligent Systems.
We thank the MPI-IS Software Workshop for their thoughtful feedback and support during the project refactoring.
NICE Toolbox © 2026 Max-Planck-Gesellschaft zur Förderung der Wissenschaften e.V. is licensed under AGPL-3.0, see LICENSE.
NICE Toolbox has optional support for third-party algorithms and models that are distributed under their own licenses. See LICENSES_ALGORITHMS.md for the full list.