[EHL][APL][JSL] suspend-resume with playback/capture test failure with 'rtcwake return value error'

[mengdonglin]

This error was reported by CI in daily test on 2020-09-13. Suspend and resume test with playback/capture failed within 25 iterations. dmesg shows "rtcwake return value error". But before these 2 failed test cases, CI runs check-suspend-resume-50.sh (w/o playback or capture, 50 iterations) successfully, the error cannot be reproduced in that case.

2 failed cases:
check-suspend-resume-with-playback-25.sh
check-suspend-resume-with-capture-25.sh

Test recipe:
Start Time: 2020-09-13 21:26:28 UTC
End Time: 2020-09-14 01:08:25 UTC
Kernel Branch: topic/sof-dev
Kernel Commit: 02293e7
SOF Branch: master
SOF Commit:46e49cc3

OS: Ubuntu 20.04
Platform: EHL RVP w/ codec in I2S mode

Dmesg when error happens:

...
2020-09-13 23:59:58 UTC Sub-Test: [REMOTE_COMMAND] sleep for 5
2020-09-14 00:00:03 UTC Sub-Test: [REMOTE_INFO] Check for the kernel log status
2020-09-14 00:00:03 UTC Sub-Test: [REMOTE_INFO] ===== Round(14/15) =====
2020-09-14 00:00:03 UTC Sub-Test: [REMOTE_COMMAND] Run the command: rtcwake -m mem -s 5
rtcwake: wakeup from "mem" using /dev/rtc0 at Mon Sep 14 00:00:09 2020
2020-09-14 00:00:11 UTC Sub-Test: [REMOTE_COMMAND] sleep for 5
2020-09-14 00:00:16 UTC Sub-Test: [REMOTE_INFO] Check for the kernel log status
2020-09-14 00:00:16 UTC Sub-Test: [REMOTE_INFO] ===== Round(15/15) =====
2020-09-14 00:00:16 UTC Sub-Test: [REMOTE_COMMAND] Run the command: rtcwake -m mem -s 5
rtcwake: wakeup from "mem" using /dev/rtc0 at Mon Sep 14 00:00:22 2020
rtcwake: write error
2020-09-14 00:00:22 UTC Sub-Test: [REMOTE_ERROR] rtcwake return value error
2020-09-14 00:00:22 UTC Sub-Test: [REMOTE_ERROR] Starting func_exit_handler(), exit status=1, FUNCNAME stack:
2020-09-14 00:00:22 UTC Sub-Test: [REMOTE_ERROR]  die()  @  /home/ubuntu/sof-test/test-case/../case-lib/lib.sh
2020-09-14 00:00:22 UTC Sub-Test: [REMOTE_ERROR]  main()  @  /home/ubuntu/sof-test/test-case/check-suspend-resume.sh:80
2020-09-14 00:00:22 UTC Sub-Test: [REMOTE_INFO] Test Result: FAIL!
2020-09-14 00:00:22 UTC [REMOTE_ERROR] suspend resume failed
2020-09-14 00:00:22 UTC [REMOTE_ERROR] Starting func_exit_handler(), exit status=1, FUNCNAME stack:
2020-09-14 00:00:22 UTC [REMOTE_ERROR]  main()  @  /home/ubuntu/sof-test/test-case/check-suspend-resume-with-audio.sh
2020-09-14 00:00:24 UTC [REMOTE_WARNING] Process(es) started by /home/ubuntu/sof-test/test-case/check-suspend-resume-with-audio.sh are still active, kill these process(es):
2020-09-14 00:00:24 UTC [REMOTE_WARNING] Catch pid: 86606 aplay -Dhw:0,6 -r 48000 -c 2 -f S16_LE /dev/zero -q
2020-09-14 00:00:24 UTC [REMOTE_WARNING] Kill cmd:'aplay -Dhw:0,6 -r 48000 -c 2 -f S16_LE /dev/zero -q' by kill -9
2020-09-14 00:00:24 UTC [REMOTE_INFO] Test Result: FAIL!

[mengdonglin]

@marc-hb The same failure can also be reproduced on APL_UP2_NOCODEC in CI daily test on 2020-09-11.

Recipe:
Start Time: 2020-09-11 21:25:38 UTC
End Time: 2020-09-12 01:11:18 UTC
Kernel Branch: topic/sof-dev
Kernel Commit: 02293e7
SOF Branch: master
SOF Commit:64a1657b

[libinyang]

I have found such issue for weeks, which randomly happens in such test cases: check-suspend-resume-50.sh, check-suspend-resume-with-playback-25.sh, check-suspend-resume-with-capture-25.sh. And the root cause seems to be the same:
“rtcwake return value error”

And the case check-suspend-resume-50.sh only runs suspend/resume, without any audio operations. So I suppose this is a none sof related issue. I manually test such cases with run-all-tests.sh on both EHL RVP and CRB several times, and I can’t reproduce this issue.

[mengdonglin]

Same error also happens on CML Notebook with HD-A codec in stress test check-suspend-resume-500.sh