You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
On the Windows app SDK using CameraHelper, FrameArrived time execution for a period of time will not be triggered. #375
@GreenShadeZhang I don't believe any of the Camera/Media stuff is enabled in the WindowsAppSDK yet (at least to the point it's exposed in the same way that we consume in the Toolkit). We've been waiting for them to enable support for these scenarios. I know that's definitely true for the CameraPreview which uses the MediaPlayerElement, but not sure what's supported or not 100% with the underlying MediaFrameReader api the CameraHelper uses.
I know in the docs hereand here they're only calling out the CaptureCameraUI from the Windows.Media.Capture namespace, so not sure if this is a missing gap in documentation or their radar/planning.
I'd recommend opening a discussion on the Windows App SDK repo to be sure of what is and isn't supposed to be working at this time.
@GreenShadeZhang I don't believe any of the Camera/Media stuff is enabled in the WindowsAppSDK yet (at least to the point it's exposed in the same way that we consume in the Toolkit). We've been waiting for them to enable support for these scenarios. I know that's definitely true for the CameraPreview which uses the MediaPlayerElement, but not sure what's supported or not 100% with the underlying MediaFrameReader api the CameraHelper uses.
I know in the docs hereand here they're only calling out the CaptureCameraUI from the Windows.Media.Capture namespace, so not sure if this is a missing gap in documentation or their radar/planning.
I'd recommend opening a discussion on the Windows App SDK repo to be sure of what is and isn't supposed to be working at this time.
Thank you very much for your answer. About the media playback control and video preview control is not porting too difficult to become a low priority?
Thank you very much for your answer. About the media playback control and video preview control is not porting too difficult to become a low priority?
Hi @GreenShadeZhang, I'm not 100% sure I'm following what you're asking here. The Windows App SDK team handles the underlying APIs we're using, so filing an issue on their repo and asking for clarity on what should be supported is a first step. They may clarify if it's supposed to work now, not supposed to work, or something they're planning to implement in an upcoming release or not.
From there, once we know that, we can figure out on our side in the Toolkit for our next 8.0 release if this is something we can support, assuming it is supported in the platform at that time.
I've had a similar problem, I guess it can wait until the MediaPlayerElement is updated in the SDK.
Recently I replaced the camera API of UWP with opencvsharp, and the effect is OK is that the CPU occupation is too high, I don't know if you have any good way.
Thank you very much for your answer. About the media playback control and video preview control is not porting too difficult to become a low priority?
Hi @GreenShadeZhang, I'm not 100% sure I'm following what you're asking here. The Windows App SDK team handles the underlying APIs we're using, so filing an issue on their repo and asking for clarity on what should be supported is a first step. They may clarify if it's supposed to work now, not supposed to work, or something they're planning to implement in an upcoming release or not.
From there, once we know that, we can figure out on our side in the Toolkit for our next 8.0 release if this is something we can support, assuming it is supported in the platform at that time.
Thanks @GreenShadeZhang, our big hope with 8.0 is to maintain everything from a single codebase (vs. the forks/branches we have now). So, ideally we'll be able to make this easier for us to validate the behavior of and test across both UWP and the Windows App SDK.
Thanks @GreenShadeZhang, our big hope with 8.0 is to maintain everything from a single codebase (vs. the forks/branches we have now). So, ideally we'll be able to make this easier for us to validate the behavior of and test across both UWP and the Windows App SDK.
The latest 1.2 already supports media playback controls, can the latest version of the Community Toolkit test this feature, or wait for the 8.0 release.
@GreenShadeZhang I'm not sure, it's not something we've tested, but I don't think the code was commented out, so assuming the APIs map the same, then it may work?
We weren't able to validate and bring forward our existing winui branch, so we'll be starting to port features over to our new infrastructure that supports both UWP and the Windows App SDK in the near future.
If you do have a chance to try, let us know the outcome. Thanks!
I believe the issue here is that CameraHelper is calling VideoMediaFrame.GetVideoFrame without disposing the result. This code is where the issue is occurring.
Disposing the video frame after the event handler is invoked seems to work. I’m not sure if there would be an issue in doing that for API consumers, though.
Additionally, the Reader_FrameArrived should check if there are any subscribers to the CameraHelper.FrameArrived event before getting the video frame, so the common scenario of using CameraPreview, which does not rely on that event, can avoid the overhead.
I believe the issue here is that CameraHelper is calling VideoMediaFrame.GetVideoFrame without disposing the result. This code is where the issue is occurring.
Disposing the video frame after the event handler is invoked seems to work. I’m not sure if there would be an issue in doing that for API consumers, though.
Additionally, the Reader_FrameArrived should check if there are any subscribers to the CameraHelper.FrameArrived event before getting the video frame, so the common scenario of using CameraPreview, which does not rely on that event, can avoid the overhead.
Yesterday I located the cause of the problem. I found that it was still a problem with the release of unmanaged objects. I will test more today to see if I can solve this problem. If so, I will file a PR.
Hi Everyone, I am still encountering an issue using the latest RC build, 8.1.240606-rc. My preview freezes after 400-500 frames. From what I can tell from Source Link, I cannot dispose the VideoFrame in my FrameArrived event handler because the getter on FrameEventArgs makes a copy of the original.
If the fix is not included, that the only way for me to get the original VideoFrame from FrameEventArgs would be to reflect out the internal field _videoFrame and dispose of it in my handler
If there is another workaround for this issue or if I am doing something wrong?
Activity
michael-hawker commentedon Jul 19, 2022
@GreenShadeZhang I don't believe any of the Camera/Media stuff is enabled in the WindowsAppSDK yet (at least to the point it's exposed in the same way that we consume in the Toolkit). We've been waiting for them to enable support for these scenarios. I know that's definitely true for the
CameraPreviewwhich uses theMediaPlayerElement, but not sure what's supported or not 100% with the underlyingMediaFrameReaderapi the CameraHelper uses.I know in the docs here and here they're only calling out the
CaptureCameraUIfrom theWindows.Media.Capturenamespace, so not sure if this is a missing gap in documentation or their radar/planning.I'd recommend opening a discussion on the Windows App SDK repo to be sure of what is and isn't supposed to be working at this time.
GreenShadeZhang commentedon Jul 22, 2022
Thank you very much for your answer. About the media playback control and video preview control is not porting too difficult to become a low priority?
michael-hawker commentedon Jul 22, 2022
Hi @GreenShadeZhang, I'm not 100% sure I'm following what you're asking here. The Windows App SDK team handles the underlying APIs we're using, so filing an issue on their repo and asking for clarity on what should be supported is a first step. They may clarify if it's supposed to work now, not supposed to work, or something they're planning to implement in an upcoming release or not.
From there, once we know that, we can figure out on our side in the Toolkit for our next 8.0 release if this is something we can support, assuming it is supported in the platform at that time.
AlexanderBlackman commentedon Aug 17, 2022
I've had a similar problem, I guess it can wait until the MediaPlayerElement is updated in the SDK.
GreenShadeZhang commentedon Aug 17, 2022
Recently I replaced the camera API of UWP with opencvsharp, and the effect is OK is that the CPU occupation is too high, I don't know if you have any good way.
GreenShadeZhang commentedon Sep 26, 2022
Recently I found out that the Windows app SDK 1.2 preview version supports MediaPlayerElement, so I feel that 8.0 can test the integrity of this feature.
https://learn.microsoft.com/en-us/windows/apps/windows-app-sdk/preview-channel#winui-3
In addition, I also saw some precautions for camera data processing in the document, and I don't know if I need to pay attention to these things in the windows app SDK.
https://learn.microsoft.com/en-us/windows/uwp/audio-video-camera/process-media-frames-with-mediaframereader
michael-hawker commentedon Sep 26, 2022
Thanks @GreenShadeZhang, our big hope with 8.0 is to maintain everything from a single codebase (vs. the forks/branches we have now). So, ideally we'll be able to make this easier for us to validate the behavior of and test across both UWP and the Windows App SDK.
GreenShadeZhang commentedon Sep 27, 2022
Sounds very good and very exciting.
GreenShadeZhang commentedon Dec 7, 2022
The latest 1.2 already supports media playback controls, can the latest version of the Community Toolkit test this feature, or wait for the 8.0 release.
michael-hawker commentedon Dec 7, 2022
@GreenShadeZhang I'm not sure, it's not something we've tested, but I don't think the code was commented out, so assuming the APIs map the same, then it may work?
We weren't able to validate and bring forward our existing winui branch, so we'll be starting to port features over to our new infrastructure that supports both UWP and the Windows App SDK in the near future.
If you do have a chance to try, let us know the outcome. Thanks!
BreeceW commentedon Mar 18, 2024
I believe the issue here is that CameraHelper is calling
VideoMediaFrame.GetVideoFramewithout disposing the result. This code is where the issue is occurring.Disposing the video frame after the event handler is invoked seems to work. I’m not sure if there would be an issue in doing that for API consumers, though.
Additionally, the
Reader_FrameArrivedshould check if there are any subscribers to theCameraHelper.FrameArrivedevent before getting the video frame, so the common scenario of usingCameraPreview, which does not rely on that event, can avoid the overhead.GreenShadeZhang commentedon Mar 23, 2024
Can this problem really be solved? I think this function can reduce a lot of code for people who use it for video processing.
@michael-hawker
GreenShadeZhang commentedon Mar 26, 2024
Yesterday I located the cause of the problem. I found that it was still a problem with the release of unmanaged objects. I will test more today to see if I can solve this problem. If so, I will file a PR.
ThreeSevenths commentedon Jul 5, 2024
Hi Everyone, I am still encountering an issue using the latest RC build, 8.1.240606-rc. My preview freezes after 400-500 frames. From what I can tell from Source Link, I cannot dispose the
VideoFramein myFrameArrivedevent handler because the getter onFrameEventArgsmakes a copy of the original.Can someone confirm:
VideoFramefromFrameEventArgswould be to reflect out the internal field_videoFrameand dispose of it in my handlerMy Window code looks like this below:
MainWindow.xaml.cs
MainWindow.xaml