Skip to content

Extend custom Processor API - #95

Open
C-Achard wants to merge 17 commits into
cy/expand-coverage-july26from
cy/extend-processor-api
Open

Extend custom Processor API#95
C-Achard wants to merge 17 commits into
cy/expand-coverage-july26from
cy/extend-processor-api

Conversation

@C-Achard

@C-Achard C-Achard commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Scope

Introduces improved custom processors in the GUI, especially focusing on how processor instances are created, configured, notified of recording events, and cleaned up.
Adds support for passing detailed recording context to processors, allows processors to opt into being built in the worker process, and improves logging and error handling throughout the lifecycle.

This gives the processors far broader capabilities:

  • Can operate on context passed from the GUI, e.g. recording file paths
  • Can execute custom logic specifically on recording start/stop
  • May be constructed the same way as the 1.0 version of live-GUI for legacy compat
  • Adds much better logging re: custom processor creation and operation

TODO:

  • Reset DLC_LIFECYCLE_EXTRA_LOGS to False in config.py.

Automated summary

Processor lifecycle and context management:

  • Adds hooks in main_window.py to notify processors when recording starts and stops, passing a detailed context dictionary, and provides a generic method to save processor data on stop (_notify_processor_recording_started, _notify_processor_recording_stopped, _save_processor_data_if_available, _build_processor_recording_context) [1] [2].
  • Updates BaseProcessorSocket to support context-aware saving, including new methods for setting/getting recording context and save paths, and implements on_recording_started and on_recording_stopped hooks [1] [2].
  • The processor save() method now uses the context-provided path if available and logs more informative messages.

Processor instantiation and configuration:

  • Introduces ProcessorSpec dataclass to represent processor construction parameters, and utilities to create specs and determine if a processor should be built in the worker (create_spec_from_scan, processor_builds_in_worker) [1] [2].
  • Modifies processor instantiation logic in main_window.py to use specs for worker-built processors and to log context at key points.
  • Updates dlc_processor.py to support configuration with either a processor instance or a spec, to track which was used, and to clean up processor instances on reset/shutdown [1] [2] [3] [4].

Logging and debugging improvements:

  • Adds a new DLC_LIFECYCLE_EXTRA_LOGS config flag and a log_processor_context utility to emit detailed lifecycle logs for debugging processor handling [1] [2].
  • Enables extra logging around processor instantiation, worker startup, and cleanup [1] [2].

Processor API enhancements:

  • BaseProcessorSocket gains the experimental PROCESSOR_BUILD_IN_WORKER attribute to signal where it should be constructed.
  • The stop() method in BaseProcessorSocket now optionally saves data before stopping.

These changes together make the processor workflow more robust, flexible, and debuggable, especially for advanced/experimental processor plugins and custom user extensions.

@C-Achard C-Achard self-assigned this Jul 10, 2026
@C-Achard C-Achard added enhancement New feature or request processor Custom DLC-live processors: either base class, public API, docs or new custom examples labels Jul 10, 2026
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 137010e to c5f039a Compare July 14, 2026 09:23
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from c5f039a to 2d2d529 Compare July 14, 2026 09:44
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 2d2d529 to 3cfff63 Compare July 16, 2026 09:47
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 3cfff63 to ead4210 Compare July 16, 2026 11:15
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from ead4210 to 330cbc9 Compare August 10, 2026 12:11
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 330cbc9 to f224128 Compare August 10, 2026 14:27
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch 2 times, most recently from c301cb4 to 95f9f75 Compare August 11, 2026 09:24
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 95f9f75 to 8e818c4 Compare August 11, 2026 11:56
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 8e818c4 to 5933d93 Compare August 11, 2026 13:23
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 5933d93 to 2392272 Compare August 12, 2026 08:16
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 2392272 to 3fb756e Compare August 12, 2026 09:31
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 3fb756e to 7b2522c Compare August 12, 2026 13:50
@C-Achard
C-Achard marked this pull request as ready for review August 17, 2026 13:27
@C-Achard
C-Achard requested a review from deruyter92 August 17, 2026 13:28
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 0eb3499 to 27f7cad Compare August 18, 2026 13:16
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 27f7cad to 633cadc Compare August 18, 2026 13:46
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch 2 times, most recently from c781904 to d7bf98b Compare August 18, 2026 14:20
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from d7bf98b to 81cf5a0 Compare August 18, 2026 14:24
Add deferred processor construction via `ProcessorSpec` so processors that require worker-thread context can be built in `DLCLiveProcessor._worker_loop` instead of the GUI thread. Update both main window and service configuration paths to choose between immediate instantiation and worker-side specs, add lifecycle/context logging helpers, and improve processor shutdown/reset cleanup by calling processor `stop()` when appropriate. Also adjust DLC logging defaults to reduce timing noise while enabling targeted lifecycle diagnostics.
RecordingManager now captures and exposes a current/last recording file context, including run/session directories plus per-camera video and timestamp sidecar paths, so downstream processor hooks can still resolve finalized files after stop_all(). VideoRecorder adds explicit output_path and timestamp_json_path properties, and timestamp saving now reuses the shared timestamp path accessor.
Wire the main window to notify custom DLC processors when recording starts/stops, with a shared recording context (run dir, filename stem, and file metadata). Refactor processor lookup into a helper and only fall back to generic save() if no stop hook handles persistence.

Extend BaseProcessorSocket with recording context/save-path state, start/stop hook methods, and a stop(save=...) option. Update save() to use an explicit path or configured default path, create parent directories, and improve logging for skipped/failed saves.
Expand custom processor test coverage around recording context and save-path behavior. This adds a new test module for BaseProcessorSocket and DLCLiveMainWindow recording hook interactions, including optional hook handling and processor lookup paths.

The existing base processor tests were also cleaned up to use pytest `tmp_path` for file outputs instead of writing into module data directories, remove manual cleanup blocks, and tighten assertions/formatting for save and recording flows.
Replaced the service’s direct dependency on `BaseProcessorSocket.do_build_in_worker` with `processor_builds_in_worker` from `processor_utils`, improving reuse and reducing coupling to socket internals. Also updated the recording context docstring to use `timestamp_json_files` for consistent key naming.
Store the processor recording context when recording starts and reuse that same context when notifying `on_recording_stopped`, instead of rebuilding it from recorder state. This also centralizes stop-time processor finalization (stop hook + save) and invokes it both on normal async stop and during shutdown when a recording context exists.
Adjust shutdown/reset flow to avoid cleaning up the processor while the worker thread is still alive. Cleanup now runs immediately only after a successful stop, and is deferred to the reaper path when a pending reset completes after the worker eventually joins.
Corrects the recording-stop cleanup path to clear `_processor_recording_context` instead of an unrelated attribute. This ensures the async stop handler sees the finalized state consistently and avoids stale recording context after shutdown.
Refines recording lifecycle integration by capturing finalized recording file context when notifying `on_recording_stopped`, and introducing tracking for recording-start notifications during DLC initialization. It also tightens processor teardown behavior by distinguishing pending reset vs pending plugin cleanup, and ensures full processor plugin cleanup when stopping DLC from the UI/service.
Improve processor metadata finalization by rebuilding recording context from the current run directory and then overlaying cached values, so missing fields are backfilled before merging file context. Also update socket processor saving so explicitly provided relative filenames are saved under the `data/` directory, matching expected output behavior.
Adds focused tests around processor recording context handling in `DLCLiveMainWindow`, including file metadata propagation, stop-time context refresh, and optional processor hooks. It also introduces a regression test for `BaseProcessorSocket.save()` to ensure explicit relative filenames still write under the legacy `data/` directory. UI label testing for unknown camera IDs was relaxed to assert the neutral text is present rather than requiring an exact string.
Set `_processor_recording_started_notified` immediately after a successful recording-start hook callback so the UI tracks notification state correctly and avoids repeated notifications. Also add a unit test that verifies `ProcessorSpec` processors are instantiated on the `DLCLiveWorker` thread, wired into `DLCLive`, and properly stopped/cleared when resetting the processor.
@C-Achard
C-Achard force-pushed the cy/extend-processor-api branch from 81cf5a0 to fab8afd Compare August 18, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request processor Custom DLC-live processors: either base class, public API, docs or new custom examples

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants