Extend custom Processor API - #95
Open
C-Achard wants to merge 17 commits into
Open
Conversation
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
July 14, 2026 09:23
137010e to
c5f039a
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
July 14, 2026 09:44
c5f039a to
2d2d529
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
July 16, 2026 09:47
2d2d529 to
3cfff63
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
July 16, 2026 11:15
3cfff63 to
ead4210
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 10, 2026 12:11
ead4210 to
330cbc9
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 10, 2026 14:27
330cbc9 to
f224128
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
2 times, most recently
from
August 11, 2026 09:24
c301cb4 to
95f9f75
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 11, 2026 11:56
95f9f75 to
8e818c4
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 11, 2026 13:23
8e818c4 to
5933d93
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 12, 2026 08:16
5933d93 to
2392272
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 12, 2026 09:31
2392272 to
3fb756e
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 12, 2026 13:50
3fb756e to
7b2522c
Compare
C-Achard
marked this pull request as ready for review
August 17, 2026 13:27
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 18, 2026 13:16
0eb3499 to
27f7cad
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 18, 2026 13:46
27f7cad to
633cadc
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
2 times, most recently
from
August 18, 2026 14:20
c781904 to
d7bf98b
Compare
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 18, 2026 14:24
d7bf98b to
81cf5a0
Compare
Add deferred processor construction via `ProcessorSpec` so processors that require worker-thread context can be built in `DLCLiveProcessor._worker_loop` instead of the GUI thread. Update both main window and service configuration paths to choose between immediate instantiation and worker-side specs, add lifecycle/context logging helpers, and improve processor shutdown/reset cleanup by calling processor `stop()` when appropriate. Also adjust DLC logging defaults to reduce timing noise while enabling targeted lifecycle diagnostics.
RecordingManager now captures and exposes a current/last recording file context, including run/session directories plus per-camera video and timestamp sidecar paths, so downstream processor hooks can still resolve finalized files after stop_all(). VideoRecorder adds explicit output_path and timestamp_json_path properties, and timestamp saving now reuses the shared timestamp path accessor.
Wire the main window to notify custom DLC processors when recording starts/stops, with a shared recording context (run dir, filename stem, and file metadata). Refactor processor lookup into a helper and only fall back to generic save() if no stop hook handles persistence. Extend BaseProcessorSocket with recording context/save-path state, start/stop hook methods, and a stop(save=...) option. Update save() to use an explicit path or configured default path, create parent directories, and improve logging for skipped/failed saves.
Expand custom processor test coverage around recording context and save-path behavior. This adds a new test module for BaseProcessorSocket and DLCLiveMainWindow recording hook interactions, including optional hook handling and processor lookup paths. The existing base processor tests were also cleaned up to use pytest `tmp_path` for file outputs instead of writing into module data directories, remove manual cleanup blocks, and tighten assertions/formatting for save and recording flows.
Replaced the service’s direct dependency on `BaseProcessorSocket.do_build_in_worker` with `processor_builds_in_worker` from `processor_utils`, improving reuse and reducing coupling to socket internals. Also updated the recording context docstring to use `timestamp_json_files` for consistent key naming.
Store the processor recording context when recording starts and reuse that same context when notifying `on_recording_stopped`, instead of rebuilding it from recorder state. This also centralizes stop-time processor finalization (stop hook + save) and invokes it both on normal async stop and during shutdown when a recording context exists.
Adjust shutdown/reset flow to avoid cleaning up the processor while the worker thread is still alive. Cleanup now runs immediately only after a successful stop, and is deferred to the reaper path when a pending reset completes after the worker eventually joins.
Corrects the recording-stop cleanup path to clear `_processor_recording_context` instead of an unrelated attribute. This ensures the async stop handler sees the finalized state consistently and avoids stale recording context after shutdown.
Refines recording lifecycle integration by capturing finalized recording file context when notifying `on_recording_stopped`, and introducing tracking for recording-start notifications during DLC initialization. It also tightens processor teardown behavior by distinguishing pending reset vs pending plugin cleanup, and ensures full processor plugin cleanup when stopping DLC from the UI/service.
Improve processor metadata finalization by rebuilding recording context from the current run directory and then overlaying cached values, so missing fields are backfilled before merging file context. Also update socket processor saving so explicitly provided relative filenames are saved under the `data/` directory, matching expected output behavior.
Adds focused tests around processor recording context handling in `DLCLiveMainWindow`, including file metadata propagation, stop-time context refresh, and optional processor hooks. It also introduces a regression test for `BaseProcessorSocket.save()` to ensure explicit relative filenames still write under the legacy `data/` directory. UI label testing for unknown camera IDs was relaxed to assert the neutral text is present rather than requiring an exact string.
Set `_processor_recording_started_notified` immediately after a successful recording-start hook callback so the UI tracks notification state correctly and avoids repeated notifications. Also add a unit test that verifies `ProcessorSpec` processors are instantiated on the `DLCLiveWorker` thread, wired into `DLCLive`, and properly stopped/cleared when resetting the processor.
C-Achard
force-pushed
the
cy/extend-processor-api
branch
from
August 18, 2026 14:27
81cf5a0 to
fab8afd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
Introduces improved custom processors in the GUI, especially focusing on how processor instances are created, configured, notified of recording events, and cleaned up.
Adds support for passing detailed recording context to processors, allows processors to opt into being built in the worker process, and improves logging and error handling throughout the lifecycle.
This gives the processors far broader capabilities:
TODO:
DLC_LIFECYCLE_EXTRA_LOGSto False inconfig.py.Automated summary
Processor lifecycle and context management:
main_window.pyto notify processors when recording starts and stops, passing a detailed context dictionary, and provides a generic method to save processor data on stop (_notify_processor_recording_started,_notify_processor_recording_stopped,_save_processor_data_if_available,_build_processor_recording_context) [1] [2].BaseProcessorSocketto support context-aware saving, including new methods for setting/getting recording context and save paths, and implementson_recording_startedandon_recording_stoppedhooks [1] [2].save()method now uses the context-provided path if available and logs more informative messages.Processor instantiation and configuration:
ProcessorSpecdataclass to represent processor construction parameters, and utilities to create specs and determine if a processor should be built in the worker (create_spec_from_scan,processor_builds_in_worker) [1] [2].main_window.pyto use specs for worker-built processors and to log context at key points.dlc_processor.pyto support configuration with either a processor instance or a spec, to track which was used, and to clean up processor instances on reset/shutdown [1] [2] [3] [4].Logging and debugging improvements:
DLC_LIFECYCLE_EXTRA_LOGSconfig flag and alog_processor_contextutility to emit detailed lifecycle logs for debugging processor handling [1] [2].Processor API enhancements:
BaseProcessorSocketgains the experimentalPROCESSOR_BUILD_IN_WORKERattribute to signal where it should be constructed.stop()method inBaseProcessorSocketnow optionally saves data before stopping.These changes together make the processor workflow more robust, flexible, and debuggable, especially for advanced/experimental processor plugins and custom user extensions.