[DNM][WiP][PoC] userspace LL scheduling: LLEXT & multicore - #10945
Draft
lyakh wants to merge 30 commits into
Draft
[DNM][WiP][PoC] userspace LL scheduling: LLEXT & multicore#10945lyakh wants to merge 30 commits into
lyakh wants to merge 30 commits into
Conversation
Merged
lyakh
force-pushed
the
llext-ull
branch
2 times, most recently
from
July 14, 2026 09:58
a462fcf to
9f6770d
Compare
This was referenced Jul 14, 2026
This was referenced Aug 12, 2026
Place the pipeline position lookup table in the sysuser memory partition and replace k_spinlock with a dynamically allocated k_mutex when CONFIG_SOF_USERSPACE_LL is enabled. Spinlocks disable interrupts which is a privileged operation unavailable from user-mode threads. The mutex pointer is stored in a separate APP_SYSUSER_BSS variable outside the SHARED_DATA struct so Zephyr's kernel object tracking can recognize it for syscall verification. Move pipeline_posn_init() from task_main_start() to primary_core_init() before platform_init(), so the mutex is allocated before ipc_user_init() grants thread access to it. In pipeline_posn_get(), bypass the sof_get() kernel singleton and access the shared structure directly when running in user-space. Grant the ipc_user_init thread access to the pipeline position mutex via new pipeline_posn_grant_access() helper. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
Collaborator
Author
|
After multiple PRs got merged we're down from 108 commits to 32 |
Make the options from app/overlays/ptl/ll_userspace_overlay.conf the default for the Intel Panther Lake (ptl) and Wildcat Lake (wcl) build targets, so user-space Low-Latency audio pipelines are enabled without having to pass the overlay explicitly. As noted in the overlay header, once user-space LL is enabled for a target by default the settings belong in the SOF board file directly. For ptl the board already provides the user-space base (USERSPACE, dynamic threads, MMU L2 tables, domain partitions), so only the LL overlay options are added and the conflicting telemetry / cold-store / llext / modules defaults are flipped to match the overlay. wcl had no user-space base at all; since CONFIG_SOF_USERSPACE_LL depends on CONFIG_USERSPACE it would otherwise be silently dropped. Mirror ptl's user-space base into the wcl board file as well so LL actually takes effect there. The ll_userspace_overlay.conf file is kept unchanged; it now re-applies identical values and remains usable by development build scripts. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
.cold and .coldrodata partitions can be empty, avoid a failure in such cases. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Add a missing header for the zephyr_ll_(un)lock_sched() functions. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Extract a privileged LLEXT-related part from lib_manager_module_create() into a separate function to be called from kernel context. At the same time lib_manager_mod_free_priv() already executes privileged operations; to make it callable in userspace, convert lib_manager_free_module() to a system call. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Use a type-cast instead of a memcpy() to obtain struct ipc4_module_init_instance data. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
scheduler_get_data() only finds scheduler data for kernel mode schedulers. Add a similar function for userspace schedulers. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
ipc4_init_module_instance() should be called when CONFIG_SOF_USERSPACE_LL isn't selected but also when initializing a DP module. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
If LL runs in userspace, it needs access to loaded LLEXT modules, running in DP more too. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Add two syscall functions to allocate and map, and to unmap vregion for userspace modules. For now only used for DP modules. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
vregion_get(), vregion_put() and vregion_set_interim() should also be callable from the userspace. Make them syscalls. Also remove redundant symbol exporting since the vregion API shouldn't be used directly by LLEXT modules. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
The DP scheduler runs tasks in userspace mode, it's registered with the user scheduler list, therefore it should use scheduler_get_user_data(), not scheduler_get_data(). Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
scheduler_dp_ll_tick(() has to recalculate DP deadlines and reschedule DP threads. Make it a syscall to be able to call it from the userspace LL scheduler. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Make scheduler_dp_internal_free() a syscall in the "application" DP implementation. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Make scheduling LL thread and synchronisation objects per-core and forward IPCs and scheduling events accordingly. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
In case of userspace LL scheduling the (also userspace) IPC thread needs access rights to DP assets like the thread itself and its stack and synchronisation primitives. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
The LL userspace thread has to interact with the DP one. Grant required rights. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Switching to the userspace mode in DP and LL cases differs. Add a comment to explain that. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Join several needlessly broken lines. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Extract common initialisation code from comp_new_ipc4_user() and comp_new_ipc4() into a new function. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Move llext_manager_add_partition() and llext_manager_rm_partition() higher in the file for future use. No functional change. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
With userspace LL enabled module freeing runs in syscall contex on behalf of the userspace IPC thread. That thread doesn't have access to DRAM. Therefore we cannot call lib_manager_get_module_manifest() in that case. Use SRAM module data by calling llext_manager_mod_find() instead. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
llext_manager_unload_module() is called in a syscall context on behalf of the userspace IPC thread, so it doesn't have direct access to LLEXT module DRAM data. Map section headers temporarily for the duration of the function. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
llext_manager_free_module() cannot access DRAM when running with userspace LL enabled. Don't call lib_manager_get_library_manifest() to obtain the DRAM descriptor, needed to verify the entry index. The index is now verified by llext_manager_mod_find(). Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When running in syscall context on behalf of a userspace thread dynamically mapped memory doesn't automatically become accessible. To make it accessible it has to be added to the thread memory domain. This is a problem for loadable modules with executable cold sections. To be able to execute them they have to be mapped to threads with the executable bit set. While for linking that memory has to be mapped writable. To solve the problem we perform linking from the kernel IPC context before forwarding to the userspace IPC thread. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
LLEXT is now working with userspace LL and can be enabled. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
This reverts commit b647695127b6c23f4ebeb510f3cbd0c08a23dc24.
CONFIG_COLD_STORE_EXECUTE_DRAM can now be re-enabled for userspace LL. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Make vregion_alloc(), vregion_alloc_coherent(), vregion_alloc_align(), vregion_alloc_coherent_align(), and vregion_free() available as Zephyr system calls for user-space threads. Add K_SYSCALL_MEMORY_WRITE verification to all syscall handlers to validate the calling thread has access to the vregion's managed memory area. Add CONFIG_SOF_USERSPACE_INTERFACE_VREGION Kconfig option to control the feature. It is auto-selected by SOF_USERSPACE_LL when SOF_VREGIONS is enabled. Signed-off-by: Kai Vehmanen <kai.vehmanen@linux.intel.com>
The DP scheduler can now be user with userspace LL. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
This was referenced Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This includes #10558 and my patches on top to enable LLEXT and multicore. Current status: passes simple tests with nocodec with both core 0 and core 1 streaming. 2 streams simultaneously run into a problem when the first of them terminates. WiP.