Bootup issue on Talos KBDev with 6.19-rc8

[vnarapar]

Bootup issue observed on Talos KBDev with 6.19-rc8

Full logs are here: https://lava-oss.qualcomm.com/scheduler/job/33197#L1576
[ 244.878944][ T74] INFO: task swapper/0:1 blocked for more than 120 seconds.
[ 244.886342][ T74] Not tainted 6.19.0-rc8-00567-gc10784363e6a #1
[ 244.893160][ T74] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 244.901926][ T74] task:swapper/0 state:D stack:10416 pid:1 tgid:1 ppid:0 task_flags:0x0140 flags:0x00000010
[ 244.913832][ T74] Call trace:
[ 244.917082][ T74] __switch_to+0x108/0x1c8 (T)
[ 244.921861][ T74] __schedule+0x2fc/0x740
[ 244.926189][ T74] schedule+0x38/0xb8
[ 244.930158][ T74] schedule_timeout+0x130/0x148
[ 244.935020][ T74] __wait_for_common+0xec/0x1f8
[ 244.940071][ T74] wait_for_completion+0x28/0x40
[ 244.945016][ T74] __flush_work+0x8c/0x110
[ 244.949409][ T74] flush_work+0x38/0x50
[ 244.953533][ T74] deferred_probe_initcall+0x64/0xc8
[ 244.958814][ T74] do_one_initcall+0x60/0x2c0
[ 244.963474][ T74] do_initcalls+0xfc/0x1f8
[ 244.967868][ T74] kernel_init_freeable+0x13c/0x298
[ 244.973059][ T74] kernel_init+0x2c/0x1f8
[ 244.977367][ T74] ret_from_fork+0x10/0x20

[sgaud-quic]

Issue is seen with 6.19 as well, Tingwei Zhang (@quic-tingweiz) can you please check this
https://lava-oss.qualcomm.com/scheduler/job/40837

[changmil]

After enabled some debug logs, Dmesg logs shows that smem start probing, but stuck without bond successfully.
From the call stack, it stuck at smem_dram_parse+0x240.

[   25.121238][    T1] calling  deferred_probe_initcall+0x0/0xc8 @ 1
[   25.128133][   T68] platform 86000000.smem: Retrying from deferred list
[   25.135626][   T68] platform 86000000.smem: bus: 'platform': __driver_probe_device: matched device with driver qcom-smem
[   25.146850][   T68] platform 86000000.smem: bus: 'platform': really_probe: probing driver qcom-smem with device
[  251.295094][   T74] INFO: task swapper/0:1 blocked for more than 120 seconds.
[  251.302488][   T74]       Not tainted 6.19.0-00713-ge56f4349ce1d-dirty #17
[  251.309571][   T74] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  251.318339][   T74] task:swapper/0       state:D stack:10440 pid:1     tgid:1     ppid:0      task_flags:0x0140 flags:0x00000010
[  251.330251][   T74] Call trace:
[  251.333505][   T74]  __switch_to+0x108/0x1c8 (T)
[  251.338287][   T74]  __schedule+0x310/0x7a8
[  251.342616][   T74]  schedule+0x38/0xb8
[  251.346587][   T74]  schedule_timeout+0x130/0x148
[  251.351459][   T74]  __wait_for_common+0xec/0x1f8
[  251.356560][   T74]  wait_for_completion+0x28/0x40
[  251.361518][   T74]  __flush_work+0x8c/0x110
[  251.365916][   T74]  flush_work+0x38/0x50
[  251.370042][   T74]  deferred_probe_initcall+0x64/0xc8
[  251.375331][   T74]  do_one_initcall+0x60/0x2c0
[  251.379990][   T74]  do_initcalls+0xfc/0x1f8
[  251.384387][   T74]  kernel_init_freeable+0x13c/0x298
[  251.389583][   T74]  kernel_init+0x2c/0x1f8
[  251.393888][   T74]  ret_from_fork+0x10/0x20
…
[  251.478519][    C6] CPU: 6 UID: 0 PID: 68 Comm: kworker/u32:1 Not tainted 6.19.0-00713-ge56f4349ce1d-dirty #17 PREEMPT
[  251.478523][    C6] Hardware name: Qualcomm Technologies, Inc. QCS615 Ride (IQ-615 Beta EVK) (DT)
[  251.478526][    C6] Workqueue: events_unbound deferred_probe_work_func
[  251.478534][    C6] pstate: 40400005 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  251.478538][    C6] pc : smem_dram_parse+0x240/0x270
[  251.478543][    C6] lr : smem_dram_parse+0x23c/0x270
[  251.478546][    C6] sp : ffff8000803c3a80
[  251.478548][    C6] x29: ffff8000803c3a90 x28: ffff000085e2d0a0 x27: ffff80008084d000
[  251.478551][    C6] x26: 000000000000fffe x25: 0000000000000000 x24: 0000000000000030
[  251.478555][    C6] x23: 0000000000000020 x22: 000000000000000a x21: 000000000000000b
[  251.478558][    C6] x20: ffff000084199c10 x19: ffff800081007340 x18: 00000000ffffffff
[  251.478561][    C6] x17: ffffbaaa2343b464 x16: ffffbaaa2343b2ac x15: ffffbaaa2343aea8
[  251.478564][    C6] x14: ffffbaaa2343eca4 x13: ffffbaaa22a3d924 x12: ffffbaaa22a3d0e0
[  251.478567][    C6] x11: ffffbaaa2343a9fc x10: ffffbaaa234397c8 x9 : ffffbaaa23ba6b2c
[  251.478570][    C6] x8 : ffff8000803c3918 x7 : 0000000000000000 x6 : 0000000000000001
[  251.478573][    C6] x5 : ffffbaaa24f1b000 x4 : ffffbaaa24f1b390 x3 : 0000000000000000
[  251.478576][    C6] x2 : ffff0000840ca540 x1 : ffff0000840ca540 x0 : ffff0000a55d7480
[  251.478579][    C6] Call trace:
[  251.478580][    C6]  smem_dram_parse+0x240/0x270 (P)
[  251.478584][    C6]  qcom_smem_probe+0x518/0x758
[  251.478588][    C6]  platform_probe+0x64/0xa8
[  251.478593][    C6]  really_probe+0xc8/0x3f0
[  251.478597][    C6]  __driver_probe_device+0xdc/0x190
[  251.478600][    C6]  driver_probe_device+0x44/0x120
[  251.478604][    C6]  __device_attach_driver+0xc4/0x178
[  251.478608][    C6]  bus_for_each_drv+0x90/0xf8
[  251.478611][    C6]  __device_attach+0xa8/0x1d8
[  251.478615][    C6]  device_initial_probe+0x58/0x68
[  251.478619][    C6]  bus_probe_device+0x40/0xb8
[  251.478622][    C6]  deferred_probe_work_func+0xbc/0x128
[  251.478626][    C6]  process_one_work+0x180/0x450
[  251.478631][    C6]  worker_thread+0x26c/0x388
[  251.478635][    C6]  kthread+0x120/0x140
[  251.478639][    C6]  ret_from_fork+0x10/0x20

it stuck at a dead loop.
The machine code 0x14000000 means jump to current address.
This leads to dead loop.

ffff800080a04cc8: 14000000     	b	0xffff800080a04cc8 <smem_dram_parse+0x240>

This dead loop is abnormal and cannot be found from the C code.
After some investigation we found it is a TRAP from compiler to deal with an array out of bounds path.
https://github.com/qualcomm-linux/kernel/blob/qcom-next/drivers/soc/qcom/smem_dramc.c

struct dentry *smem_dram_parse(struct device *dev)
{
                case INFO_V3_WITH_14_FREQS:
                                smem_dram_parse_v3_data(dram, data, true); <= @@@Here input additional_freq_entry as true**
                                break;
…
static void smem_dram_parse_v3_data(struct smem_dram *dram, void *data, bool additional_freq_entry)
{
                /* This may be 13 or 14 */
                int num_freq_entries = MAX_DDR_FREQ_NUM_V3;  <= @@@Here MAX_DDR_FREQ_NUM_V3 define to 13
                struct ddr_details_v3 *details = data;
 
                if (additional_freq_entry)
                                num_freq_entries++;  <= @@@additional_freq_entry =ture and here makes num_freq_entries to 14
 
                for (int i = 0; i < num_freq_entries; i++) {
                                struct ddr_freq_table *freq_entry = &details->ddr_freq_tbl.ddr_freq[i]; <= @@@Here will have possibility to access ddr_freq_tbl.ddr_freq[13], out-of-bounds happen.
 
                                if (freq_entry->freq_khz && freq_entry->enabled)
                                                dram->frequencies[dram->num_frequencies++] = 1000 * freq_entry->freq_khz;
                }
}

/* V3 */
struct ddr_freq_plan_v3 {
                struct ddr_freq_table ddr_freq[MAX_DDR_FREQ_NUM_V3]; /* NOTE: some have 14 like v5 */ <= @@@but this definition makes the valid array id 0-12
                u8 num_ddr_freqs;
                phys_addr_t clk_period_address;
};

This is a 100% out of bounds issue from code path.
So the compiler treated it to a dead loop as TRAP which caused the stuck issue.

also found below warning logs when compiling:

In function ‘smem_dram_parse_v3_data’,
    inlined from ‘smem_dram_parse’ at /local/mnt/workspace/changmil/kernel-6.19-rc6/drivers/soc/qcom/smem_dramc.c:265:3:
/local/mnt/workspace/changmil/kernel-6.19-rc6/drivers/soc/qcom/smem_dramc.c:167:31: warning: iteration 13 invokes undefined behavior [-Waggressive-loop-optimizations]
  167 |                 if (freq_entry->freq_khz && freq_entry->enabled)
      |                     ~~~~~~~~~~^~~~~~~~~~
/local/mnt/workspace/changmil/kernel-6.19-rc6/drivers/soc/qcom/smem_dramc.c:164:27: note: within this loop
  164 |         for (int i = 0; i < num_freq_entries; i++) {
      |                         ~~^~~~~~~~~~~~~~~~~~
  CC      drivers/soc/qcom/smsm.o
  CC      kernel/module/main.o

[sgaud-quic]

Closing this as issue is fixed and not seen on latest tag : https://github.com/qualcomm-linux/kernel/tree/qcom-next-7.0-rc2-20260306