Repository navigation
Closed
Labels
CMLApplies to Comet Lake platformApplies to Comet Lake platformP1Blocker bugs or important featuresBlocker bugs or important featuresbugSomething isn't working as expectedSomething isn't working as expected
Description
suspend_stress_test will fail at 2nd iteration.
kernel: topic/cros-sof-v4.19 branch 8492020
fw: master 8483480
topology: sof-cml-rt5682 (with PR #976)
Platform: CML Chromebook
In fact, there are some error at the 1st resume:
[ 8221.557905] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1000010 timedout
[ 8221.557920] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8221.557932] sof-audio-pci 0000:00:1f.3: error: status = 0x00000000 panic = 0x00000000
[ 8221.557960] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8221.557983] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010f0f successful
[ 8221.558688] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f successful
[ 8221.558701] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 0 : core_mask f
[ 8221.558719] sof-audio-pci 0000:00:1f.3: error: Error code=0xffffffff: FW status=0xffffffff
[ 8221.558739] sof-audio-pci 0000:00:1f.3: error: iteration 0 of Core En/ROM load failed: -62
[ 8221.558763] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f0f0f successful
[ 8221.558778] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f0f0e successful
[ 8221.558787] sof-audio-pci 0000:00:1f.3: unstall/run core: core_mask = 1
[ 8221.558799] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 1 : core_mask 1
[ 8221.559673] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x80000000 successful
[ 8221.560674] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010e0e successful
[ 8221.560770] sof-audio-pci 0000:00:1f.3: error: rx list empty but received 0x0
[ 8221.560784] sof-audio-pci 0000:00:1f.3: error: can't find message header 0x0
And similar error at the 2nd suspend, finally leads to suspend fail.
[ 8261.402649] sof-audio-pci 0000:00:1f.3: error: rx list empty but received 0x0
[ 8261.402667] sof-audio-pci 0000:00:1f.3: error: can't find message header 0x0
[ 8261.472783] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1000010 timedout
[ 8261.472806] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8261.472819] sof-audio-pci 0000:00:1f.3: error: status = 0x00000000 panic = 0x00000000
[ 8261.472841] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8261.472870] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010f0f successful
[ 8261.472897] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f successful
[ 8261.472917] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 0 : core_mask f
[ 8261.472939] sof-audio-pci 0000:00:1f.3: error: Error code=0xffffffff: FW status=0xffffffff
[ 8261.472953] sof-audio-pci 0000:00:1f.3: error: iteration 0 of Core En/ROM load failed: -62
[ 8261.473552] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f0f0f successful
[ 8261.473577] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f0f0e successful
[ 8261.473592] sof-audio-pci 0000:00:1f.3: unstall/run core: core_mask = 1
[ 8261.473614] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 1 : core_mask 1
[ 8261.474544] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x80000000 successful
[ 8261.474554] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010e0e successful
[ 8261.474628] sof-audio-pci 0000:00:1f.3: error: rx list empty but received 0x0
[ 8261.474640] sof-audio-pci 0000:00:1f.3: error: can't find message header 0x0
[ 8261.508129] nvme 0000:01:00.0: pci_pm_suspend+0x0/0x1a1 returned 0 after 120266 usecs
[ 8261.544624] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1000010 timedout
[ 8261.544646] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8261.544658] sof-audio-pci 0000:00:1f.3: error: status = 0x00000000 panic = 0x00000000
[ 8261.544676] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8261.544690] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010f0f successful
[ 8261.545503] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f successful
[ 8261.545513] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 0 : core_mask f
[ 8261.545544] sof-audio-pci 0000:00:1f.3: error: Error code=0xffffffff: FW status=0xffffffff
[ 8261.545561] sof-audio-pci 0000:00:1f.3: error: iteration 1 of Core En/ROM load failed: -62
[ 8261.545588] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f0f0f successful
[ 8261.545614] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f0f0e successful
[ 8261.545624] sof-audio-pci 0000:00:1f.3: unstall/run core: core_mask = 1
[ 8261.545647] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 1 : core_mask 1
[ 8261.546500] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x80000000 successful
[ 8261.546516] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010e0e successful
[ 8261.546580] sof-audio-pci 0000:00:1f.3: error: rx list empty but received 0x0
[ 8261.546592] sof-audio-pci 0000:00:1f.3: error: can't find message header 0x0
[ 8261.616459] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1000010 timedout
[ 8261.616475] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8261.616498] sof-audio-pci 0000:00:1f.3: error: status = 0x00000000 panic = 0x00000000
[ 8261.616517] sof-audio-pci 0000:00:1f.3: unknown ROM status value 05000001
[ 8261.616549] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1010f0f successful
[ 8261.616569] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0xf0f successful
[ 8261.616588] sof-audio-pci 0000:00:1f.3: DSP core(s) enabled? 0 : core_mask f
[ 8261.616609] sof-audio-pci 0000:00:1f.3: error: Error code=0xffffffff: FW status=0xffffffff
[ 8261.616622] sof-audio-pci 0000:00:1f.3: error: iteration 2 of Core En/ROM load failed: -62
[ 8261.616638] sof-audio-pci 0000:00:1f.3: error: dsp init failed after 3 attempts with err: -62
[ 8261.616691] sof-audio-pci 0000:00:1f.3: unknown ROM status value ffffffff
[ 8261.616715] sof-audio-pci 0000:00:1f.3: error: status = 0xffffffff panic = 0xffffffff
[ 8261.616740] sof-audio-pci 0000:00:1f.3: unknown ROM status value ffffffff
[ 8261.616751] sof-audio-pci 0000:00:1f.3: error: failed to reset DSP
[ 8261.616765] sof-audio-pci 0000:00:1f.3: error: failed to boot DSP firmware after resume -62
[ 8261.616862] sof-audio-pci 0000:00:1f.3: ipc tx: 0x40010000
[ 8261.923451] sof-audio-pci 0000:00:1f.3: error: ipc timed out for 0x40010000 size 0x4c
[ 8261.923474] sof-audio-pci 0000:00:1f.3: unknown ROM status value 01009701
[ 8261.923489] sof-audio-pci 0000:00:1f.3: error: status = 0x001d003c panic = 0x00000000
[ 8261.923506] sof-audio-pci 0000:00:1f.3: unknown ROM status value 01009701
[ 8261.923537] sof-audio-pci 0000:00:1f.3: error: ctx_save ipc error during suspend -110
[ 8261.973456] sof-audio-pci 0000:00:1f.3: FW Poll Status: reg=0x1d003c timedout
[ 8261.973469] sof-audio-pci 0000:00:1f.3: error: reset enter failed: core_mask f adspcs 0x1d003c
[ 8261.973486] sof-audio-pci 0000:00:1f.3: error: dsp core reset failed: core_mask f
[ 8261.973500] sof-audio-pci 0000:00:1f.3: error: failed to power down core during suspend
[ 8261.973514] sof-audio-pci 0000:00:1f.3: error: suspending dsp
[ 8261.973531] sof-audio-pci 0000:00:1f.3: error: failed to power down DSP during suspend -5
The issue won't be solved by thesofproject/linux#651 which will prevent reading reply of CTX_SAVE
Metadata
Metadata
Assignees
Labels
CMLApplies to Comet Lake platformApplies to Comet Lake platformP1Blocker bugs or important featuresBlocker bugs or important featuresbugSomething isn't working as expectedSomething isn't working as expected
Activity
mengdonglin commentedon Feb 18, 2019
@xun2z please help to reproduce this issue.
emilchudzik commentedon Feb 18, 2019
Hi,
I try to reproduce issue on WHL RVP + I2S, but I have issue with entering suspend: #1019
In spite of that FW and tplg loaded properly, aplay works.
kernel: topic/sof-dev: branch 69a41dab
FW: 2a86314
topology: sof-cnl-nocodec.tplg
On this kernel on dmesg there is no msg from CTX_SAVE/CTX_RESTORE. Is it excpected?
bardliao commentedon Feb 19, 2019
@emilchudzik
You will see something like "ipc tx: 0x40010000: GLB_PM_MSG: CTX_SAVE" if CONFIG_SND_SOC_SOF_DEBUG_VERBOSE_IPC is set in kernel config.
You can set it from make menuconfig
xun2z commentedon Feb 19, 2019
Test env:
sof cml-001-drop-stable: 1f5a965
kernel topic/cros-sof-v4.19: f69d61898a
tplg: sof-cml-rt5682.tplg
Test env:
sof cml-002-drop-stable: 2a86314
kernel topic/cros-sof-v4.19: 8492020d5967
tplg: sof-cml-rt5682.tplg
Test env:
sof: master dcd825b
kernel topic/sof-dev 69a41da
tplg: sof-cml-rt5682.tplg
Though ctx_save error occurs, whl will not be blocked.
dmesg_v4.19.log
dmesg_v4.20.log
emilchudzik commentedon Feb 19, 2019
I have this option enabled.
emilchudzik commentedon Feb 19, 2019
Due to reboot issue #662 on WHL RVP+I2S instead of suspend I use another WHL to find a repro.
On WHL RVP + HDA reworks I can playback on HDA (codec ALC700) and enter suspend/resume in loop. It works fine on each iteration.
kernel: cros-sof-v4.19 / 8492020d5967
FW: cml-002-drop-stable / dcd825b
tplg for hda: sof-hda-generic.tplg
But I see in dmesg error with ctx:
@bardliao could you check what happend in this function:
error: ipc error for 0x40010000 size 0xc
log from last iteration: playback/suspend/resume
dmesg3.txt
bardliao commentedon Feb 20, 2019
@emilchudzik Please see #984 And it can be fixed by thesofproject/linux#651
emilchudzik commentedon Feb 20, 2019
I have made check with patch thesofproject/linux#651. And playback and suspend works fine in loop. There is no error message with it.
Platform: WHL_RVP
kernel: cros-sof-v4.19 / 8492020d5967
FW: cml-002-drop-stable / dcd825b
tplg for hda: sof-hda-generic.tplg
I also try to check as Bard ask - on the same platform using nocodec mode.
I prepare suitable kernel and topology. Then trying to go suspend cause OS freezing all the time.
G3 on platform restore booting system again.
4 remaining items
ranj063 commentedon Feb 28, 2019
@plbossart @emilchudzik @bardliao this simple change in the kernel fixes the issue for me but it was simply a trial and error on my part. The value of 70ms timeout was based on the skylake driver and it worked in GLK.
I could use some feedback on how to determine the correct value to be set.
diff --git a/sound/soc/sof/intel/hda.h b/sound/soc/sof/intel/hda.h
index 54b817358e31..ee4817abc251 100644
--- a/sound/soc/sof/intel/hda.h
+++ b/sound/soc/sof/intel/hda.h
@@ -198,7 +198,7 @@
#define HDA_DSP_RESET_TIMEOUT 50
#define HDA_DSP_BASEFW_TIMEOUT 3000
#define HDA_DSP_INIT_TIMEOUT 500
-#define HDA_ROM_INIT_TIMEOUT 70
+#define HDA_ROM_INIT_TIMEOUT 100
#define HDA_DSP_CTRL_RESET_TIMEOUT 100
#define HDA_DSP_WAIT_TIMEOUT 500 /* 500 msec /
#define HDA_DSP_REG_POLL_INTERVAL_US 500 / 0.5 msec */
ranj063 commentedon Feb 28, 2019
@lbetlej @mmaka1 could you please comment on the value for the ROM init timeout value?
lbetlej commentedon Mar 4, 2019
I suggest at least 300 ms for HDA_ROM_INIT_TIMEOUT.
emilchudzik commentedon Mar 5, 2019
I have just checked flow from this issue topic.
Platform: Hatch
kernel: cros-sof-v4.19 / 8492020d5967 + thesofproject/linux#651.
FW: cml-002-drop-stable / dcd825b
tplg: cml-002-drop-stable / dcd825b sof-cml-rt5682.zip
Commands "suspend_stress_test" in my opion return fail to suspend (see below screenshot).
When I use more than 2 iter (I mean paramneter: -c2) -> unable to enter suspend.
When I use -c1 several times -> platform trying to go suspend: screen is off, but ssh is still active. So it seems like script "suspend_stress_test" didn't enter platform to suspend even in first iterartion.
Is there any options to go suspend in "user natural way"? e.g. pressing button on platform to double check if it is platform or script issue?
Short pressing "pwr" button try to power off.
emilchudzik commentedon Mar 5, 2019
I didn't see that thinks or WHL RVP.
Here I use: rtcwake command or short pressing power button.
Both method properly entering suspend WHL RVP.
ranj063 commentedon Mar 6, 2019
@lbetlej we used to have this value set to 300ms before. But the suspend stress test failed and after following the analysis done fro skylake, it was decided to use 70ms. I might leave it at 70 for GLK and only change it to 300ms for CML.
ranj063 commentedon Mar 8, 2019
thesofproject/linux#704 fixes this issue.
@emilchudzik could you please check if you can run a stress test with this change in the kernel
suspend_stress_test -c 500
lgirdwood commentedon Mar 11, 2019
@emilchudzik did thesofproject/linux#704 fix it for you ? If so we should close.
emilchudzik commentedon Mar 13, 2019
on Hatch this patch doesn't help. I have still issue with first suspend, suspending doesn't work -> screen is off for a while, but logs return:
Suspend 1 of 1: sleep for 7 seconds...Suspend failed, s0ix count did not increment from 0)
emilchudzik commentedon Mar 13, 2019
I suggest to check repro on CML (@bardliao as submitter) and if doesn't exists then closed it.
I can rise new issue that on Hatch (CML CHromebook) suspending doesn't work at whole using scripts:
suspend_stress_test
ranj063 commentedon Mar 13, 2019
@emilchudzik I'm going to send you the latest coreboot that is needed for suspend stress test to work. Please give it a try
Tom-0505 commentedon Sep 23, 2020
@ranj063 can you send me too, I have met the same problem as @emilchudzik too.
THX
Tom-0505 commentedon Sep 23, 2020
@emilchudzik Is this issue solved? I've met the same problem, and I've turned on HIPM,DIPM,DEVSLP, still can't solved the problem. Please kindly teach me how to do.
THX