igt@xe_pat@pt-caching on shard-bmg@xe-pw-164469v1 Test details

Detail	Value
Duration	unknown
Hostname	shard-bmg-1
Igt-Version	IGT-Version: 2.4-g2a9f1c1ae (x86_64) (Linux: 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ x86_64)
Out	Using IGT_SRANDOM=1775600643 for randomisation Opened device: /dev/dri/card0 Test requirement not met in function __igt_unique____real_main2210, file ../tests/intel/xe_pat.c:2298: Test requirement: intel_graphics_ver(dev_id) == IP_VER(35, 10) Last errno: 2, No such file or directory Starting subtest: pt-caching va_bits: 48, num_objs: 16384, spread over pt levels: [0]: 16384 [1]: 512 [2]: 16 runner: This test was killed due to a kernel taint (0x4244). runner: This test was killed due to exceeding disk usage limit. (Used 10985486 bytes, limit 10485760) This test caused an abort condition: Kernel badly tainted (0x4244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened.
Err	Starting subtest: pt-caching Received signal SIGQUIT. Stack trace: #0 [fatal_sig_handler+0x17b] #1 [__sigaction+0x50] #2 [ioctl+0x3b] #3 [drmIoctl+0x30] #4 [___xe_bo_create+0xe4] #5 [__xe_bo_create_caching+0x17] #6 [xe_bo_create_caching+0x27] #7 [pt_create_objects+0x19a] #8 [pt_caching_test+0x14b] #9 [__igt_unique____real_main2210+0xbbe] #10 [main+0x3a] #11 [__libc_init_first+0x8a] #12 [__libc_start_main+0x8b] #13 [_start+0x25]
Dmesg Scroll to first warning	<6> [237.054433] Console: switching to colour dummy device 80x25 <6> [237.054728] [IGT] xe_pat: executing <6> [237.064123] [IGT] xe_pat: starting subtest pt-caching <6> [239.819892] xe 0000:03:00.0: [drm] PL2 disabled for channel 0, val 0x00000000 <7> [239.822909] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x2d2e2b2b <7> [239.823070] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x2e2d2e2d <4> [242.681930] ------------[ cut here ]------------ <4> [242.681946] xe 0000:03:00.0: [drm] Tile0: GT0: Unexpected engine class:instance 3:8 for context utilization <4> [242.681959] WARNING: drivers/gpu/drm/xe/xe_lrc.c:2580 at xe_lrc_timestamp+0x196/0x460 [xe], CPU#6: kworker/u64:50/3229 <4> [242.682451] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [242.682797] efi_pstore nfnetlink autofs4 <4> [242.682821] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [242.682837] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [L]=SOFTLOCKUP <4> [242.682843] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [242.682850] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [242.682878] RIP: 0010:xe_lrc_timestamp+0x1a7/0x460 [xe] <4> [242.683381] Code: b6 79 26 48 89 55 c8 88 45 d0 e8 84 e9 5c e1 48 89 c6 48 8d 3d ea ba 37 00 0f b6 4d d0 41 54 45 89 e9 45 0f b6 c7 48 8b 55 c8 <67> 48 0f b9 3a 58 eb 91 41 0f b6 bd 99 0f 00 00 e8 14 e0 03 00 41 <4> [242.683390] RSP: 0018:ffffc900071c3c00 EFLAGS: 00010246 <4> [242.683402] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [242.683410] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1004f50 <4> [242.683416] RBP: ffffc900071c3c90 R08: 0000000000000000 R09: 0000000000000003 <4> [242.683422] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000008 <4> [242.683428] R13: 0000000000000003 R14: ffff888124321580 R15: 0000000000000000 <4> [242.683435] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [242.683443] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [242.683451] CR2: 00007c34ac404ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [242.683459] PKRU: 55555554 <4> [242.683465] Call Trace: <4> [242.683471] <TASK> <4> [242.683484] ? enable_work+0x9d/0x100 <4> [242.683509] ? xe_lrc_start_seqno+0x33/0x70 [xe] <4> [242.683955] guc_exec_queue_timedout_job+0xf80/0x2400 [xe] <4> [242.684352] ? lock_sync+0x100/0x100 <4> [242.684374] ? lock_release+0xd0/0x2b0 <4> [242.684403] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [242.684432] process_one_work+0x239/0x760 <4> [242.684466] worker_thread+0x200/0x3f0 <4> [242.684481] ? __pfx_worker_thread+0x10/0x10 <4> [242.684494] kthread+0x10d/0x150 <4> [242.684511] ? __pfx_kthread+0x10/0x10 <4> [242.684530] ret_from_fork+0x3d4/0x480 <4> [242.684542] ? __pfx_kthread+0x10/0x10 <4> [242.684559] ret_from_fork_asm+0x1a/0x30 <4> [242.684598] </TASK> <4> [242.684605] irq event stamp: 391209 <4> [242.684612] hardirqs last enabled at (391215): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [242.684629] hardirqs last disabled at (391220): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [242.684641] softirqs last enabled at (390404): [<ffffffff82616c8e>] neigh_periodic_work+0x27e/0x360 <4> [242.684656] softirqs last disabled at (390400): [<ffffffff82616a48>] neigh_periodic_work+0x38/0x360 <4> [242.684670] ---[ end trace 0000000000000000 ]--- <7> [242.684683] xe 0000:03:00.0: [drm:guc_exec_queue_timedout_job [xe]] Tile0: GT0: Check job timeout: seqno=39338, lrc_seqno=39338, guc_id=0, running_time_ms=233719, timeout_ms=5000, diff=0xfebc30b1 <6> [243.326009] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x209 <5> [243.326132] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39338, lrc_seqno=39338, guc_id=0, flags=0x73 in no process [-1] <7> [243.326158] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [243.326549] ------------[ cut here ]------------ <4> [243.326557] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [243.326565] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [243.327040] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [243.327374] efi_pstore nfnetlink autofs4 <4> [243.327401] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [243.327420] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [243.327427] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [243.327436] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [243.327467] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [243.327857] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [243.327867] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [243.327906] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [243.327915] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [243.327921] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [243.327929] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [243.327936] R13: ffff888104ab7410 R14: ffff888123af0000 R15: 00000000ffffffc2 <4> [243.327945] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [243.327954] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [243.327962] CR2: 00007c34ac404ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [243.327970] PKRU: 55555554 <4> [243.327978] Call Trace: <4> [243.327985] <TASK> <4> [243.328007] ? lock_sync+0x100/0x100 <4> [243.328034] ? __pfx_autoremove_wake_function+0x10/0x10 <4> [243.328063] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [243.328093] process_one_work+0x239/0x760 <4> [243.328126] worker_thread+0x200/0x3f0 <4> [243.328140] ? __pfx_worker_thread+0x10/0x10 <4> [243.328153] kthread+0x10d/0x150 <4> [243.328170] ? __pfx_kthread+0x10/0x10 <4> [243.328191] ret_from_fork+0x3d4/0x480 <4> [243.328203] ? __pfx_kthread+0x10/0x10 <4> [243.328221] ret_from_fork_asm+0x1a/0x30 <4> [243.328260] </TASK> <4> [243.328268] irq event stamp: 392065 <4> [243.328275] hardirqs last enabled at (392071): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [243.328291] hardirqs last disabled at (392076): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [243.328304] softirqs last enabled at (391320): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [243.328318] softirqs last disabled at (391311): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [243.328331] ---[ end trace 0000000000000000 ]--- <6> [243.328342] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [243.328739] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [243.329372] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [243.329821] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [243.331165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [243.331461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [243.331552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [243.331637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [243.331724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [243.331813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [243.331934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [243.332021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [243.332099] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [243.332193] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [243.332303] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [243.333546] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [243.347985] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 13ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [243.348548] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [243.352997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [243.353369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [243.353713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [243.354086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [243.354416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [243.354744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [243.355104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [243.355419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [243.355715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [243.356033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [243.356324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [243.356599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [243.356863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [243.357177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [243.357455] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [243.357716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [243.357977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [243.358226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [243.358473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [243.358725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [243.358993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [243.359237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [243.359478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [243.359703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [243.359928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [243.360146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [243.360361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [243.360579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [243.360782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [243.360997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [243.361196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [243.361388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [243.361592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [243.361787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [243.361987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [243.362173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [243.362354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [243.362525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [243.362688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [243.362843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [243.363031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [243.363192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [243.363358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [243.363514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [243.363669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [243.363812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [243.363965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [243.364102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [243.364242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [243.364375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [243.364507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [243.364638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [243.364765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [243.364892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [243.365015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [243.365140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [243.365268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [243.365397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [243.365516] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [243.366211] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <7> [243.367186] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [243.367356] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [248.826518] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39338, lrc_seqno=39338, guc_id=0, flags=0x73 in no process [-1] <7> [248.826547] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [248.826998] ------------[ cut here ]------------ <4> [248.827010] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [248.827022] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:50/3229 <4> [248.827572] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [248.828260] efi_pstore nfnetlink autofs4 <4> [248.828305] CPU: 8 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [248.828331] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [248.828343] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [248.828357] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [248.828401] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [248.828990] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [248.829006] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [248.829029] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [248.829042] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [248.829051] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [248.829058] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [248.829064] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [248.829071] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [248.829079] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [248.829086] CR2: 00007d00b01d8858 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [248.829094] PKRU: 55555554 <4> [248.829100] Call Trace: <4> [248.829106] <TASK> <4> [248.829125] ? lock_sync+0x100/0x100 <4> [248.829151] ? lock_release+0xd0/0x2b0 <4> [248.829181] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [248.829211] process_one_work+0x239/0x760 <4> [248.829243] worker_thread+0x200/0x3f0 <4> [248.829257] ? __pfx_worker_thread+0x10/0x10 <4> [248.829270] kthread+0x10d/0x150 <4> [248.829285] ? __pfx_kthread+0x10/0x10 <4> [248.829304] ret_from_fork+0x3d4/0x480 <4> [248.829315] ? __pfx_kthread+0x10/0x10 <4> [248.829332] ret_from_fork_asm+0x1a/0x30 <4> [248.829370] </TASK> <4> [248.829376] irq event stamp: 398657 <4> [248.829382] hardirqs last enabled at (398663): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [248.829398] hardirqs last disabled at (398668): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [248.829410] softirqs last enabled at (398496): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [248.829423] softirqs last disabled at (398489): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [248.829435] ---[ end trace 0000000000000000 ]--- <6> [248.829446] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [248.829824] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [248.829894] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [248.830333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [248.830931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [248.831383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [248.831846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [248.832378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [248.832806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [248.833618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [248.834355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [248.835024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [248.835664] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [248.836464] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [248.837228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [248.839240] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [248.850310] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [248.850556] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [248.851923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [248.852013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [248.852090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [248.852165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [248.852238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [248.852310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [248.852382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [248.852453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [248.852525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [248.852596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [248.852668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [248.852739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [248.852809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [248.852896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [248.852975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [248.853056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [248.853139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [248.853221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [248.853302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [248.853389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [248.853477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [248.853562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [248.853647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [248.853729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [248.853812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [248.853904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [248.853987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [248.854076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [248.854163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [248.854248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [248.854332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [248.854418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [248.854507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [248.854595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [248.854686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [248.854771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [248.854857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [248.854940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [248.855022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [248.855107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [248.855195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [248.855280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [248.855362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [248.855443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [248.855529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [248.855611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [248.855694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [248.855778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [248.855865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [248.855948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [248.856028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [248.856112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [248.856193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [248.856274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [248.856353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [248.856435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [248.856514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [248.856593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [248.856677] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [248.856796] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <7> [248.857735] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [248.857846] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [253.945128] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39338, lrc_seqno=39338, guc_id=0, flags=0x73 in no process [-1] <7> [253.945156] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [253.945545] ------------[ cut here ]------------ <4> [253.945553] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [253.945561] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [253.946095] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [253.946435] efi_pstore nfnetlink autofs4 <4> [253.946462] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [253.946479] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [253.946487] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [253.946496] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [253.946525] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [253.946937] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [253.946948] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [253.946961] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [253.946971] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [253.946979] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [253.946987] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [253.946995] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [253.947004] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [253.947014] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [253.947023] CR2: 000000c000436000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [253.947032] PKRU: 55555554 <4> [253.947040] Call Trace: <4> [253.947047] <TASK> <4> [253.947070] ? lock_sync+0x100/0x100 <4> [253.947096] ? lock_release+0xd0/0x2b0 <4> [253.947126] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [253.947157] process_one_work+0x239/0x760 <4> [253.947192] worker_thread+0x200/0x3f0 <4> [253.947208] ? __pfx_worker_thread+0x10/0x10 <4> [253.947222] kthread+0x10d/0x150 <4> [253.947239] ? __pfx_kthread+0x10/0x10 <4> [253.947259] ret_from_fork+0x3d4/0x480 <4> [253.947272] ? __pfx_kthread+0x10/0x10 <4> [253.947292] ret_from_fork_asm+0x1a/0x30 <4> [253.947332] </TASK> <4> [253.947340] irq event stamp: 402075 <4> [253.947347] hardirqs last enabled at (402081): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [253.947364] hardirqs last disabled at (402086): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [253.947376] softirqs last enabled at (401330): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.947391] softirqs last disabled at (401313): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.947405] ---[ end trace 0000000000000000 ]--- <5> [253.950640] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39393, lrc_seqno=39393, guc_id=0, flags=0x73 in no process [-1] <7> [253.950644] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [253.950702] ------------[ cut here ]------------ <4> [253.950704] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [253.950705] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:31/3209 <4> [253.950773] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [253.950837] efi_pstore nfnetlink autofs4 <4> [253.950841] CPU: 8 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [253.950844] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [253.950845] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [253.950846] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [253.950852] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [253.950922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [253.950924] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [253.950927] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [253.950928] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [253.950929] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [253.950931] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [253.950932] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [253.950933] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [253.950935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [253.950936] CR2: 00007d00b01d8858 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [253.950938] PKRU: 55555554 <4> [253.950939] Call Trace: <4> [253.950940] <TASK> <4> [253.950944] ? lock_sync+0x100/0x100 <4> [253.950949] ? lock_release+0xd0/0x2b0 <4> [253.950954] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [253.950960] process_one_work+0x239/0x760 <4> [253.950966] worker_thread+0x200/0x3f0 <4> [253.950969] ? __pfx_worker_thread+0x10/0x10 <4> [253.950971] kthread+0x10d/0x150 <4> [253.950974] ? __pfx_kthread+0x10/0x10 <4> [253.950978] ret_from_fork+0x3d4/0x480 <4> [253.950980] ? __pfx_kthread+0x10/0x10 <4> [253.950983] ret_from_fork_asm+0x1a/0x30 <4> [253.950991] </TASK> <4> [253.950992] irq event stamp: 175481 <4> [253.950993] hardirqs last enabled at (175487): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [253.950996] hardirqs last disabled at (175492): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [253.950999] softirqs last enabled at (173168): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.951001] softirqs last disabled at (173163): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.951004] ---[ end trace 0000000000000000 ]--- <6> [253.951006] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [253.951070] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [253.951077] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [253.951125] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [253.951324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [253.951405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [253.951488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [253.951566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [253.951644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [253.951722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [253.951802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [253.951898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [253.951981] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [253.952105] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [253.952261] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [253.953506] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [253.964297] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [253.964617] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [253.965991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [253.966114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [253.966232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [253.966352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [253.966471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [253.966590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [253.966708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [253.966836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [253.966956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [253.967075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [253.967193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [253.967312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [253.967430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [253.967549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [253.967667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [253.967786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [253.967910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [253.968030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [253.968148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [253.968274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [253.968399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [253.968521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [253.968645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [253.968767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [253.968894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [253.969017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [253.969139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [253.969264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [253.969390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [253.969514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [253.969639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [253.969763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [253.969897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [253.970028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [253.970158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [253.970285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [253.970411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [253.970534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [253.970658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [253.970780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [253.970910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [253.971035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [253.971158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [253.971280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [253.971407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [253.971530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [253.971653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [253.971775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [253.971917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [253.972010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [253.972096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [253.972183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [253.972264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [253.972343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [253.972421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [253.972498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [253.972574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [253.972648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [253.972729] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [253.973018] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [253.973023] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39393, lrc_seqno=39393, guc_id=0, flags=0x73 in no process [-1] <7> [253.973026] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [253.973087] ------------[ cut here ]------------ <4> [253.973088] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [253.973089] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [253.973162] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [253.973226] efi_pstore nfnetlink autofs4 <4> [253.973231] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [253.973234] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [253.973235] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [253.973236] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [253.973242] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [253.973311] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [253.973312] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [253.973315] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [253.973316] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [253.973318] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [253.973319] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [253.973320] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [253.973321] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [253.973323] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [253.973324] CR2: 00007c34ac403c98 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [253.973326] PKRU: 55555554 <4> [253.973327] Call Trace: <4> [253.973328] <TASK> <4> [253.973332] ? lock_sync+0x100/0x100 <4> [253.973337] ? lock_release+0xd0/0x2b0 <4> [253.973343] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [253.973348] process_one_work+0x239/0x760 <4> [253.973354] worker_thread+0x200/0x3f0 <4> [253.973357] ? __pfx_worker_thread+0x10/0x10 <4> [253.973359] kthread+0x10d/0x150 <4> [253.973362] ? __pfx_kthread+0x10/0x10 <4> [253.973366] ret_from_fork+0x3d4/0x480 <4> [253.973368] ? __pfx_kthread+0x10/0x10 <4> [253.973371] ret_from_fork_asm+0x1a/0x30 <4> [253.973379] </TASK> <4> [253.973380] irq event stamp: 178607 <4> [253.973381] hardirqs last enabled at (178613): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [253.973384] hardirqs last disabled at (178618): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [253.973386] softirqs last enabled at (177784): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.973388] softirqs last disabled at (177769): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.973391] ---[ end trace 0000000000000000 ]--- <6> [253.973393] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [253.973464] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [253.973470] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [253.974772] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [253.974906] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [253.975116] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [253.975327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [253.975419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [253.975515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [253.975604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [253.975692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [253.975779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [253.975888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [253.975979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [253.976065] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [253.976159] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [253.976275] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [253.977432] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [253.987830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [253.988078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [253.989035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [253.989113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [253.989188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [253.989267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [253.989344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [253.989420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [253.989494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [253.989568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [253.989642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [253.989715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [253.989791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [253.989882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [253.989957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [253.990032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [253.990106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [253.990181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [253.990254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [253.990328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [253.990400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [253.990479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [253.990558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [253.990637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [253.990719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [253.990798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [253.990885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [253.990961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [253.991035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [253.991113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [253.991190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [253.991267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [253.991344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [253.991421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [253.991504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [253.991586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [253.991667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [253.991745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [253.991825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [253.991905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [253.991985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [253.992062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [253.992142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [253.992220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [253.992297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [253.992374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [253.992454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [253.992530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [253.992606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [253.992683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [253.992764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [253.992845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [253.992922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [253.993000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [253.993075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [253.993152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [253.993228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [253.993308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [253.993387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [253.993464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [253.993544] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [253.993656] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [253.993660] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39393, lrc_seqno=39393, guc_id=0, flags=0x73 in no process [-1] <7> [253.993663] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [253.993721] ------------[ cut here ]------------ <4> [253.993723] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [253.993724] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [253.993795] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [253.993864] efi_pstore nfnetlink autofs4 <4> [253.993869] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [253.993872] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [253.993873] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [253.993874] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [253.993880] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [253.993949] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [253.993951] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [253.993953] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [253.993955] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [253.993956] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [253.993957] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [253.993958] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [253.993960] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [253.993961] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [253.993963] CR2: 00007d00b023a8d0 CR3: 0000000146ed5004 CR4: 0000000000f72ef0 <4> [253.993964] PKRU: 55555554 <4> [253.993965] Call Trace: <4> [253.993967] <TASK> <4> [253.993970] ? lock_sync+0x100/0x100 <4> [253.993975] ? lock_release+0xd0/0x2b0 <4> [253.993981] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [253.993986] process_one_work+0x239/0x760 <4> [253.993993] worker_thread+0x200/0x3f0 <4> [253.993995] ? __pfx_worker_thread+0x10/0x10 <4> [253.993998] kthread+0x10d/0x150 <4> [253.994001] ? __pfx_kthread+0x10/0x10 <4> [253.994004] ret_from_fork+0x3d4/0x480 <4> [253.994007] ? __pfx_kthread+0x10/0x10 <4> [253.994010] ret_from_fork_asm+0x1a/0x30 <4> [253.994017] </TASK> <4> [253.994019] irq event stamp: 264011 <4> [253.994020] hardirqs last enabled at (264017): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [253.994022] hardirqs last disabled at (264022): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [253.994025] softirqs last enabled at (263480): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.994027] softirqs last disabled at (263475): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.994029] ---[ end trace 0000000000000000 ]--- <5> [253.994608] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39448, lrc_seqno=39448, guc_id=0, flags=0x73 in no process [-1] <7> [253.994611] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [253.994808] ------------[ cut here ]------------ <4> [253.994810] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [253.994811] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [253.994904] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [253.994980] efi_pstore nfnetlink autofs4 <4> [253.994985] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [253.994989] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [253.994990] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [253.994992] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [253.994998] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [253.995084] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [253.995086] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [253.995089] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [253.995090] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [253.995092] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [253.995093] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [253.995094] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [253.995095] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [253.995097] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [253.995098] CR2: 00007c34ac403c98 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [253.995100] PKRU: 55555554 <4> [253.995101] Call Trace: <4> [253.995102] <TASK> <4> [253.995106] ? lock_sync+0x100/0x100 <4> [253.995111] ? lock_release+0xd0/0x2b0 <4> [253.995117] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [253.995125] process_one_work+0x239/0x760 <4> [253.995131] worker_thread+0x200/0x3f0 <4> [253.995134] ? __pfx_worker_thread+0x10/0x10 <4> [253.995136] kthread+0x10d/0x150 <4> [253.995140] ? __pfx_kthread+0x10/0x10 <4> [253.995143] ret_from_fork+0x3d4/0x480 <4> [253.995145] ? __pfx_kthread+0x10/0x10 <4> [253.995148] ret_from_fork_asm+0x1a/0x30 <4> [253.995157] </TASK> <4> [253.995159] irq event stamp: 180281 <4> [253.995160] hardirqs last enabled at (180287): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [253.995169] hardirqs last disabled at (180292): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [253.995172] softirqs last enabled at (179538): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.995177] softirqs last disabled at (179533): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [253.995180] ---[ end trace 0000000000000000 ]--- <6> [253.995182] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [253.995275] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [253.995617] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [253.995631] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [253.995739] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [253.996013] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [253.996276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [253.996412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [253.996551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [253.996683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [253.996815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [253.996959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [253.997092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [253.997225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [253.997354] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [253.997496] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [253.997660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [253.998736] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.008830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.009082] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.010103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.010183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.010259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.010337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.010416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.010489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.010566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.010643] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.010720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.010794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.010879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.010953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.011027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.011100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.011176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.011252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.011327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.011402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.011475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.011556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.011638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.011716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.011796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.011883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.011965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.012043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.012120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.012200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.012279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.012357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.012434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.012514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.012602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.012689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.012774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.012862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.012943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.013021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.013098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.013173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.013256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.013335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.013413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.013489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.013570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.013648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.013725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.013800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.013887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.013966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.014044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.014125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.014203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.014280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.014355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.014431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.014506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.014581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.014659] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.014770] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.014775] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39448, lrc_seqno=39448, guc_id=0, flags=0x73 in no process [-1] <7> [254.014778] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.015695] ------------[ cut here ]------------ <4> [254.015697] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.015699] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:50/3229 <4> [254.015774] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.015846] efi_pstore nfnetlink autofs4 <4> [254.015851] CPU: 4 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.015855] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.015856] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.015857] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.015863] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.015932] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.015933] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.015936] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.015937] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.015939] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.015940] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.015941] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.015942] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [254.015944] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.015945] CR2: 00007c34ab803dd8 CR3: 00000001b9fab003 CR4: 0000000000f72ef0 <4> [254.015947] PKRU: 55555554 <4> [254.015948] Call Trace: <4> [254.015949] <TASK> <4> [254.015953] ? lock_sync+0x100/0x100 <4> [254.015958] ? lock_release+0xd0/0x2b0 <4> [254.015963] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.015969] process_one_work+0x239/0x760 <4> [254.015976] worker_thread+0x200/0x3f0 <4> [254.015978] ? __pfx_worker_thread+0x10/0x10 <4> [254.015981] kthread+0x10d/0x150 <4> [254.015984] ? __pfx_kthread+0x10/0x10 <4> [254.015988] ret_from_fork+0x3d4/0x480 <4> [254.015990] ? __pfx_kthread+0x10/0x10 <4> [254.015993] ret_from_fork_asm+0x1a/0x30 <4> [254.016000] </TASK> <4> [254.016001] irq event stamp: 409703 <4> [254.016002] hardirqs last enabled at (409709): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.016005] hardirqs last disabled at (409714): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.016008] softirqs last enabled at (409326): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.016010] softirqs last disabled at (409321): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.016012] ---[ end trace 0000000000000000 ]--- <7> [254.016026] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.016131] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [254.016230] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.016300] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.016306] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.016716] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.016938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.017032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.017127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.017215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.017301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.017389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.017477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.017564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.017646] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.017742] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.017864] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.018875] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.028831] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.029081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.030101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.030180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.030257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.030334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.030410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.030483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.030556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.030629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.030703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.030777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.030860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.030933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.031006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.031079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.031155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.031229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.031304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.031378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.031450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.031532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.031616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.031696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.031775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.031874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.031950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.032025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.032101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.032179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.032258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.032335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.032414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.032492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.032575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.032657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.032741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.032825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.032904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.032981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.033061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.033141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.033221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.033300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.033377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.033454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.033534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.033611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.033689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.033765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.033851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.033930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.034010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.034092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.034170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.034247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.034323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.034400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.034474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.034549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.034627] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.034737] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.034742] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39448, lrc_seqno=39448, guc_id=0, flags=0x73 in no process [-1] <7> [254.034744] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.034802] ------------[ cut here ]------------ <4> [254.034803] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.034805] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.034887] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.034954] efi_pstore nfnetlink autofs4 <4> [254.034959] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.034963] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.034964] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.034966] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.034971] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.035042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.035044] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.035046] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.035048] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.035050] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.035051] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.035053] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.035054] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.035056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.035058] CR2: 00007c34ac403c98 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.035059] PKRU: 55555554 <4> [254.035061] Call Trace: <4> [254.035062] <TASK> <4> [254.035066] ? lock_sync+0x100/0x100 <4> [254.035071] ? lock_release+0xd0/0x2b0 <4> [254.035077] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.035083] process_one_work+0x239/0x760 <4> [254.035090] worker_thread+0x200/0x3f0 <4> [254.035093] ? __pfx_worker_thread+0x10/0x10 <4> [254.035095] kthread+0x10d/0x150 <4> [254.035098] ? __pfx_kthread+0x10/0x10 <4> [254.035102] ret_from_fork+0x3d4/0x480 <4> [254.035104] ? __pfx_kthread+0x10/0x10 <4> [254.035108] ret_from_fork_asm+0x1a/0x30 <4> [254.035115] </TASK> <4> [254.035117] irq event stamp: 183493 <4> [254.035118] hardirqs last enabled at (183499): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.035121] hardirqs last disabled at (183504): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.035123] softirqs last enabled at (182584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.035129] softirqs last disabled at (182579): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.035131] ---[ end trace 0000000000000000 ]--- <5> [254.035734] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39503, lrc_seqno=39503, guc_id=0, flags=0x73 in no process [-1] <7> [254.035738] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.035910] ------------[ cut here ]------------ <4> [254.035912] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.035914] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [254.035996] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.036100] efi_pstore nfnetlink autofs4 <4> [254.036108] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.036113] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.036115] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.036117] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.036128] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.036216] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.036218] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.036222] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.036224] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.036226] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.036228] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.036230] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.036232] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.036234] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.036236] CR2: 0000773620d5c000 CR3: 0000000127fb6005 CR4: 0000000000f72ef0 <4> [254.036238] PKRU: 55555554 <4> [254.036240] Call Trace: <4> [254.036241] <TASK> <4> [254.036246] ? lock_sync+0x100/0x100 <4> [254.036253] ? lock_release+0xd0/0x2b0 <4> [254.036259] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.036266] process_one_work+0x239/0x760 <4> [254.036273] worker_thread+0x200/0x3f0 <4> [254.036276] ? __pfx_worker_thread+0x10/0x10 <4> [254.036279] kthread+0x10d/0x150 <4> [254.036283] ? __pfx_kthread+0x10/0x10 <4> [254.036287] ret_from_fork+0x3d4/0x480 <4> [254.036291] ? __pfx_kthread+0x10/0x10 <4> [254.036295] ret_from_fork_asm+0x1a/0x30 <4> [254.036304] </TASK> <4> [254.036306] irq event stamp: 412189 <4> [254.036307] hardirqs last enabled at (412195): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.036311] hardirqs last disabled at (412200): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.036315] softirqs last enabled at (411428): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.036320] softirqs last disabled at (411417): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.036323] ---[ end trace 0000000000000000 ]--- <6> [254.036327] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.036413] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.036429] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.036674] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.036780] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.036897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.037101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.037193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.037287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.037377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.037466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.037553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.037638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.037723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.037804] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.037909] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.038024] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.039038] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.048830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.049081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.050078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.050158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.050235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.050311] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.050386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.050462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.050537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.050611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.050685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.050757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.050834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.050908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.050987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.051064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.051138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.051211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.051285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.051358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.051431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.051512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.051594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.051673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.051754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.051842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.051921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.051999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.052076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.052157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.052239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.052318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.052398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.052478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.052565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.052651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.052736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.052817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.052911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.052989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.053070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.053151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.053233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.053312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.053389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.053465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.053545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.053621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.053699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.053775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.053860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.053936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.054013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.054096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.054175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.054254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.054332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.054414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.054493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.054569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.054649] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.054761] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.054766] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39503, lrc_seqno=39503, guc_id=0, flags=0x73 in no process [-1] <7> [254.054768] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.054832] ------------[ cut here ]------------ <4> [254.054833] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.054834] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [254.054906] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.054968] efi_pstore nfnetlink autofs4 <4> [254.054972] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.054975] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.054977] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.054978] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.054983] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.055054] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.055056] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.055058] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.055060] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.055061] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.055062] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.055063] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.055065] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.055066] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.055068] CR2: 0000773620d5c000 CR3: 000000012f44c002 CR4: 0000000000f72ef0 <4> [254.055069] PKRU: 55555554 <4> [254.055070] Call Trace: <4> [254.055071] <TASK> <4> [254.055075] ? lock_sync+0x100/0x100 <4> [254.055080] ? lock_release+0xd0/0x2b0 <4> [254.055085] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.055091] process_one_work+0x239/0x760 <4> [254.055097] worker_thread+0x200/0x3f0 <4> [254.055100] ? __pfx_worker_thread+0x10/0x10 <4> [254.055102] kthread+0x10d/0x150 <4> [254.055105] ? __pfx_kthread+0x10/0x10 <4> [254.055109] ret_from_fork+0x3d4/0x480 <4> [254.055111] ? __pfx_kthread+0x10/0x10 <4> [254.055114] ret_from_fork_asm+0x1a/0x30 <4> [254.055121] </TASK> <4> [254.055122] irq event stamp: 415289 <4> [254.055123] hardirqs last enabled at (415295): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.055126] hardirqs last disabled at (415300): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.055128] softirqs last enabled at (414294): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.055130] softirqs last disabled at (414287): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.055133] ---[ end trace 0000000000000000 ]--- <6> [254.055134] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.055207] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.055213] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.056473] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.056581] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.056687] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.056908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.057002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.057095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.057182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.057269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.057354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.057438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.057524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.057605] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.057702] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.057821] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.058859] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.068832] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.069083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.070102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.070182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.070258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.070334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.070408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.070482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.070556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.070629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.070705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.070781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.070865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.070941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.071015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.071089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.071162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.071236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.071309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.071382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.071456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.071538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.071619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.071697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.071776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.071861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.071938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.072013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.072088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.072167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.072245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.072326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.072407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.072486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.072569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.072653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.072736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.072815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.072907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.072983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.073060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.073135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.073214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.073291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.073367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.073442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.073525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.073606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.073685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.073761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.073841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.073916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.073991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.074071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.074147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.074223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.074298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.074374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.074452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.074529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.074608] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.074719] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.074723] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39503, lrc_seqno=39503, guc_id=0, flags=0x73 in no process [-1] <7> [254.074726] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.074783] ------------[ cut here ]------------ <4> [254.074785] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.074786] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.074865] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.074931] efi_pstore nfnetlink autofs4 <4> [254.074936] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.074939] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.074940] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.074941] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.074946] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.075018] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.075019] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.075022] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.075023] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.075025] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.075026] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.075028] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.075029] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.075031] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.075033] CR2: 00007d00b023a8d0 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.075035] PKRU: 55555554 <4> [254.075036] Call Trace: <4> [254.075037] <TASK> <4> [254.075041] ? lock_sync+0x100/0x100 <4> [254.075046] ? lock_release+0xd0/0x2b0 <4> [254.075052] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.075057] process_one_work+0x239/0x760 <4> [254.075064] worker_thread+0x200/0x3f0 <4> [254.075067] ? __pfx_worker_thread+0x10/0x10 <4> [254.075069] kthread+0x10d/0x150 <4> [254.075072] ? __pfx_kthread+0x10/0x10 <4> [254.075076] ret_from_fork+0x3d4/0x480 <4> [254.075078] ? __pfx_kthread+0x10/0x10 <4> [254.075082] ret_from_fork_asm+0x1a/0x30 <4> [254.075089] </TASK> <4> [254.075091] irq event stamp: 269959 <4> [254.075092] hardirqs last enabled at (269965): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.075095] hardirqs last disabled at (269970): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.075097] softirqs last enabled at (268964): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.075100] softirqs last disabled at (268957): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.075103] ---[ end trace 0000000000000000 ]--- <5> [254.075758] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39558, lrc_seqno=39558, guc_id=0, flags=0x73 in no process [-1] <7> [254.075764] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.075887] ------------[ cut here ]------------ <4> [254.075889] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.075890] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.075973] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.076056] efi_pstore nfnetlink autofs4 <4> [254.076062] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.076067] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.076069] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.076072] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.076079] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.076164] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.076167] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.076170] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.076171] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.076173] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.076174] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.076176] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.076178] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.076179] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.076181] CR2: 00007c34ac403c98 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.076183] PKRU: 55555554 <4> [254.076185] Call Trace: <4> [254.076187] <TASK> <4> [254.076191] ? lock_sync+0x100/0x100 <4> [254.076198] ? lock_release+0xd0/0x2b0 <4> [254.076204] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.076210] process_one_work+0x239/0x760 <4> [254.076218] worker_thread+0x200/0x3f0 <4> [254.076222] ? __pfx_worker_thread+0x10/0x10 <4> [254.076227] kthread+0x10d/0x150 <4> [254.076231] ? __pfx_kthread+0x10/0x10 <4> [254.076235] ret_from_fork+0x3d4/0x480 <4> [254.076240] ? __pfx_kthread+0x10/0x10 <4> [254.076244] ret_from_fork_asm+0x1a/0x30 <4> [254.076252] </TASK> <4> [254.076253] irq event stamp: 188851 <4> [254.076255] hardirqs last enabled at (188857): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.076260] hardirqs last disabled at (188862): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.076263] softirqs last enabled at (188090): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.076268] softirqs last disabled at (188085): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.076271] ---[ end trace 0000000000000000 ]--- <6> [254.076275] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.076350] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.076356] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.076420] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.076524] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.076629] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.076838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.076928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.077022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.077111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.077201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.077290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.077377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.077463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.077543] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.077637] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.077754] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.078762] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.088830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.089080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.090095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.090174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.090250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.090326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.090404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.090481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.090558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.090636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.090711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.090786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.090869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.090943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.091016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.091092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.091168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.091242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.091316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.091389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.091463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.091546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.091628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.091707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.091786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.091871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.091948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.092023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.092098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.092181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.092263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.092343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.092421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.092498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.092581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.092665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.092748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.092830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.092909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.092990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.093069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.093147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.093228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.093309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.093388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.093464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.093545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.093624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.093702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.093779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.093868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.093945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.094019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.094097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.094173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.094249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.094325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.094401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.094475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.094551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.094631] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.094741] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.094746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39558, lrc_seqno=39558, guc_id=0, flags=0x73 in no process [-1] <7> [254.094748] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.094808] ------------[ cut here ]------------ <4> [254.094809] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.094810] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [254.094895] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.094959] efi_pstore nfnetlink autofs4 <4> [254.094964] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.094967] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.094968] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.094969] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.094974] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.095044] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.095045] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.095048] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.095049] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.095050] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.095052] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.095053] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.095054] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.095056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.095057] CR2: 0000773620d5c000 CR3: 000000012f44c002 CR4: 0000000000f72ef0 <4> [254.095058] PKRU: 55555554 <4> [254.095060] Call Trace: <4> [254.095061] <TASK> <4> [254.095064] ? lock_sync+0x100/0x100 <4> [254.095069] ? lock_release+0xd0/0x2b0 <4> [254.095074] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.095080] process_one_work+0x239/0x760 <4> [254.095086] worker_thread+0x200/0x3f0 <4> [254.095089] ? __pfx_worker_thread+0x10/0x10 <4> [254.095091] kthread+0x10d/0x150 <4> [254.095094] ? __pfx_kthread+0x10/0x10 <4> [254.095097] ret_from_fork+0x3d4/0x480 <4> [254.095099] ? __pfx_kthread+0x10/0x10 <4> [254.095103] ret_from_fork_asm+0x1a/0x30 <4> [254.095110] </TASK> <4> [254.095111] irq event stamp: 419957 <4> [254.095113] hardirqs last enabled at (419963): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.095115] hardirqs last disabled at (419968): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.095117] softirqs last enabled at (419048): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.095120] softirqs last disabled at (419031): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.095122] ---[ end trace 0000000000000000 ]--- <6> [254.095124] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.095193] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.095198] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.096429] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.096535] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.096647] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.096856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.096949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.097042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.097130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.097217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.097303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.097388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.097474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.097557] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.097651] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.097768] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.098780] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.108829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.109080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.110087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.110173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.110248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.110325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.110402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.110477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.110553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.110631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.110709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.110784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.110874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.110948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.111023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.111097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.111170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.111244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.111317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.111389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.111457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.111530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.111604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.111677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.111752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.111826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.111901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.111976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.112045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.112120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.112195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.112268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.112340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.112411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.112487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.112564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.112640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.112712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.112783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.112867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.112949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.113021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.113092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.113163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.113235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.113304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.113377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.113446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.113514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.113582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.113653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.113721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.113788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.113873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.113950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.114029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.114103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.114174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.114243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.114310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.114381] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.114484] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.114490] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39558, lrc_seqno=39558, guc_id=0, flags=0x73 in no process [-1] <7> [254.114493] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.114552] ------------[ cut here ]------------ <4> [254.114553] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.114555] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.114627] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.114690] efi_pstore nfnetlink autofs4 <4> [254.114695] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.114698] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.114699] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.114700] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.114706] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.114775] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.114776] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.114779] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.114780] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.114781] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.114782] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.114784] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.114785] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.114787] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.114788] CR2: 00007c34ac403c98 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.114790] PKRU: 55555554 <4> [254.114791] Call Trace: <4> [254.114792] <TASK> <4> [254.114796] ? lock_sync+0x100/0x100 <4> [254.114800] ? lock_release+0xd0/0x2b0 <4> [254.114806] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.114811] process_one_work+0x239/0x760 <4> [254.114818] worker_thread+0x200/0x3f0 <4> [254.114820] ? __pfx_worker_thread+0x10/0x10 <4> [254.114826] kthread+0x10d/0x150 <4> [254.114829] ? __pfx_kthread+0x10/0x10 <4> [254.114833] ret_from_fork+0x3d4/0x480 <4> [254.114835] ? __pfx_kthread+0x10/0x10 <4> [254.114838] ret_from_fork_asm+0x1a/0x30 <4> [254.114845] </TASK> <4> [254.114846] irq event stamp: 192403 <4> [254.114848] hardirqs last enabled at (192409): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.114851] hardirqs last disabled at (192414): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.114853] softirqs last enabled at (191304): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.114855] softirqs last disabled at (191297): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.114858] ---[ end trace 0000000000000000 ]--- <5> [254.115432] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39613, lrc_seqno=39613, guc_id=0, flags=0x73 in no process [-1] <7> [254.115438] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.115715] ------------[ cut here ]------------ <4> [254.115718] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.115720] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.115817] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.115914] efi_pstore nfnetlink autofs4 <4> [254.115923] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.115928] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.115930] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.115932] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.115940] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.116032] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.116034] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.116036] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.116038] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.116040] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.116042] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.116043] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.116045] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.116046] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.116048] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.116050] PKRU: 55555554 <4> [254.116051] Call Trace: <4> [254.116053] <TASK> <4> [254.116057] ? lock_sync+0x100/0x100 <4> [254.116063] ? lock_release+0xd0/0x2b0 <4> [254.116070] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.116076] process_one_work+0x239/0x760 <4> [254.116083] worker_thread+0x200/0x3f0 <4> [254.116086] ? __pfx_worker_thread+0x10/0x10 <4> [254.116090] kthread+0x10d/0x150 <4> [254.116092] ? __pfx_kthread+0x10/0x10 <4> [254.116096] ret_from_fork+0x3d4/0x480 <4> [254.116098] ? __pfx_kthread+0x10/0x10 <4> [254.116102] ret_from_fork_asm+0x1a/0x30 <4> [254.116109] </TASK> <4> [254.116111] irq event stamp: 422701 <4> [254.116112] hardirqs last enabled at (422707): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.116115] hardirqs last disabled at (422712): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.116118] softirqs last enabled at (420584): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [254.116121] softirqs last disabled at (420582): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [254.116124] ---[ end trace 0000000000000000 ]--- <6> [254.116127] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.116207] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.116224] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.116453] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.116557] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.116663] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.116870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.116959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.117053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.117141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.117227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.117314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.117403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.117491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.117574] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.117669] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.117784] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.118801] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.128829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.129081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.130087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.130166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.130240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.130318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.130396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.130472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.130549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.130624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.130699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.130773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.130863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.130937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.131012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.131085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.131157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.131230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.131304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.131379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.131457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.131542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.131623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.131703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.131783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.131869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.131945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.132025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.132107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.132189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.132270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.132348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.132428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.132508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.132593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.132678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.132760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.132844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.132924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.133003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.133081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.133157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.133244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.133324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.133403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.133480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.133559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.133636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.133714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.133791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.133876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.133952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.134027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.134105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.134182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.134258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.134332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.134408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.134483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.134557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.134635] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.134745] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.134749] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39613, lrc_seqno=39613, guc_id=0, flags=0x73 in no process [-1] <7> [254.134752] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.134810] ------------[ cut here ]------------ <4> [254.134812] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.134813] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.134897] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.134962] efi_pstore nfnetlink autofs4 <4> [254.134966] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.134969] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.134970] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.134971] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.134977] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.135046] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.135048] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.135050] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.135051] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.135052] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.135054] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.135055] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.135056] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.135058] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.135059] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.135061] PKRU: 55555554 <4> [254.135062] Call Trace: <4> [254.135063] <TASK> <4> [254.135067] ? lock_sync+0x100/0x100 <4> [254.135072] ? lock_release+0xd0/0x2b0 <4> [254.135077] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.135083] process_one_work+0x239/0x760 <4> [254.135089] worker_thread+0x200/0x3f0 <4> [254.135092] ? __pfx_worker_thread+0x10/0x10 <4> [254.135094] kthread+0x10d/0x150 <4> [254.135097] ? __pfx_kthread+0x10/0x10 <4> [254.135100] ret_from_fork+0x3d4/0x480 <4> [254.135102] ? __pfx_kthread+0x10/0x10 <4> [254.135106] ret_from_fork_asm+0x1a/0x30 <4> [254.135113] </TASK> <4> [254.135114] irq event stamp: 425817 <4> [254.135115] hardirqs last enabled at (425823): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.135118] hardirqs last disabled at (425828): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.135120] softirqs last enabled at (424720): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.135123] softirqs last disabled at (424711): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.135125] ---[ end trace 0000000000000000 ]--- <6> [254.135127] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.135196] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.135202] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.136441] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.136545] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.136651] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.136864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.136956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.137051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.137139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.137225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.137310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.137397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.137483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.137569] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.137668] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.137785] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.138801] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.148834] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.149083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.150033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.150111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.150185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.150261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.150336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.150411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.150486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.150560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.150635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.150708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.150783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.150865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.150940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.151013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.151086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.151158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.151231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.151304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.151379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.151459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.151540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.151620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.151703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.151781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.151868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.151945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.152022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.152101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.152179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.152257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.152335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.152413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.152496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.152579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.152662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.152744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.152828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.152907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.152985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.153061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.153139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.153217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.153294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.153369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.153450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.153528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.153608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.153686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.153765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.153845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.153921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.154000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.154079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.154160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.154237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.154316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.154392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.154468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.154547] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.154658] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.154662] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39613, lrc_seqno=39613, guc_id=0, flags=0x73 in no process [-1] <7> [254.154665] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.154724] ------------[ cut here ]------------ <4> [254.154725] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.154726] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.154797] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.154867] efi_pstore nfnetlink autofs4 <4> [254.154872] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.154875] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.154876] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.154878] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.154883] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.154954] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.154955] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.154958] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.154959] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.154961] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.154962] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.154963] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.154964] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.154966] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.154967] CR2: 0000773620d5c000 CR3: 0000000146ed5006 CR4: 0000000000f72ef0 <4> [254.154969] PKRU: 55555554 <4> [254.154970] Call Trace: <4> [254.154971] <TASK> <4> [254.154975] ? lock_sync+0x100/0x100 <4> [254.154980] ? lock_release+0xd0/0x2b0 <4> [254.154986] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.154991] process_one_work+0x239/0x760 <4> [254.154998] worker_thread+0x200/0x3f0 <4> [254.155000] ? __pfx_worker_thread+0x10/0x10 <4> [254.155003] kthread+0x10d/0x150 <4> [254.155006] ? __pfx_kthread+0x10/0x10 <4> [254.155010] ret_from_fork+0x3d4/0x480 <4> [254.155012] ? __pfx_kthread+0x10/0x10 <4> [254.155015] ret_from_fork_asm+0x1a/0x30 <4> [254.155022] </TASK> <4> [254.155023] irq event stamp: 274241 <4> [254.155025] hardirqs last enabled at (274247): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.155028] hardirqs last disabled at (274252): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.155030] softirqs last enabled at (273338): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.155032] softirqs last disabled at (273331): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.155035] ---[ end trace 0000000000000000 ]--- <5> [254.155607] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39668, lrc_seqno=39668, guc_id=0, flags=0x73 in no process [-1] <7> [254.155610] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.155673] ------------[ cut here ]------------ <4> [254.155674] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.155676] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.155745] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.155807] efi_pstore nfnetlink autofs4 <4> [254.155811] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.155814] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.155815] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.155816] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.155821] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.155907] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.155909] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.155911] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.155913] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.155914] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.155916] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.155917] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.155918] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.155919] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.155921] CR2: 0000773620d5c000 CR3: 0000000146ed5006 CR4: 0000000000f72ef0 <4> [254.155922] PKRU: 55555554 <4> [254.155923] Call Trace: <4> [254.155924] <TASK> <4> [254.155928] ? lock_sync+0x100/0x100 <4> [254.155932] ? lock_release+0xd0/0x2b0 <4> [254.155938] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.155943] process_one_work+0x239/0x760 <4> [254.155949] worker_thread+0x200/0x3f0 <4> [254.155952] ? __pfx_worker_thread+0x10/0x10 <4> [254.155954] kthread+0x10d/0x150 <4> [254.155957] ? __pfx_kthread+0x10/0x10 <4> [254.155961] ret_from_fork+0x3d4/0x480 <4> [254.155963] ? __pfx_kthread+0x10/0x10 <4> [254.155966] ret_from_fork_asm+0x1a/0x30 <4> [254.155973] </TASK> <4> [254.155975] irq event stamp: 276869 <4> [254.155976] hardirqs last enabled at (276875): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.155978] hardirqs last disabled at (276880): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.155980] softirqs last enabled at (276708): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.155983] softirqs last disabled at (276697): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.155985] ---[ end trace 0000000000000000 ]--- <6> [254.156813] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <7> [254.156910] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.157014] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [254.157111] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.157118] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.157178] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.157386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.157477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.157571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.157658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.157746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.157840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.157931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.158018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.158102] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.158197] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.158312] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.159324] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.169832] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.170086] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.171093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.171182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.171258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.171334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.171410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.171488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.171566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.171644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.171721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.171795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.171885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.171964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.172040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.172116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.172190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.172264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.172338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.172411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.172486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.172569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.172651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.172729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.172808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.172901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.172981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.173061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.173139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.173219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.173298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.173376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.173455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.173533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.173617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.173700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.173784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.173879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.173959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.174039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.174119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.174197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.174277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.174355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.174432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.174507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.174586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.174662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.174739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.174820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.174919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.174998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.175074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.175153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.175233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.175313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.175391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.175470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.175546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.175620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.175698] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.175809] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.175813] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39668, lrc_seqno=39668, guc_id=0, flags=0x73 in no process [-1] <7> [254.175816] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.175900] ------------[ cut here ]------------ <4> [254.175901] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.175903] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.175975] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.176039] efi_pstore nfnetlink autofs4 <4> [254.176044] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.176047] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.176048] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.176049] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.176055] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.176125] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.176127] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.176130] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.176132] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.176133] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.176135] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.176136] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.176137] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.176139] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.176141] CR2: 00007c34ac403c98 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.176143] PKRU: 55555554 <4> [254.176144] Call Trace: <4> [254.176145] <TASK> <4> [254.176149] ? lock_sync+0x100/0x100 <4> [254.176154] ? lock_release+0xd0/0x2b0 <4> [254.176160] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.176165] process_one_work+0x239/0x760 <4> [254.176172] worker_thread+0x200/0x3f0 <4> [254.176175] ? __pfx_worker_thread+0x10/0x10 <4> [254.176177] kthread+0x10d/0x150 <4> [254.176180] ? __pfx_kthread+0x10/0x10 <4> [254.176184] ret_from_fork+0x3d4/0x480 <4> [254.176186] ? __pfx_kthread+0x10/0x10 <4> [254.176190] ret_from_fork_asm+0x1a/0x30 <4> [254.176197] </TASK> <4> [254.176199] irq event stamp: 200529 <4> [254.176200] hardirqs last enabled at (200535): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.176203] hardirqs last disabled at (200540): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.176205] softirqs last enabled at (199768): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.176208] softirqs last disabled at (199761): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.176211] ---[ end trace 0000000000000000 ]--- <6> [254.176213] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.176283] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.176289] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.177534] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.177640] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.177746] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.177959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.178051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.178147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.178234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.178320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.178404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.178493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.178584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.178667] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.178762] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.178886] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.179894] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.189843] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.190093] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.191113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.191191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.191268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.191346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.191422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.191499] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.191575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.191650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.191723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.191798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.191883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.191960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.192035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.192108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.192182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.192256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.192330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.192403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.192477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.192556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.192639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.192718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.192799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.192895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.192974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.193053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.193131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.193210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.193290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.193369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.193447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.193524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.193607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.193693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.193779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.193871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.193952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.194029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.194107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.194183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.194264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.194342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.194420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.194495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.194574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.194650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.194727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.194802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.194887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.194968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.195047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.195128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.195205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.195282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.195358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.195435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.195511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.195591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.195674] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.195786] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.195790] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39668, lrc_seqno=39668, guc_id=0, flags=0x73 in no process [-1] <7> [254.195792] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.196710] ------------[ cut here ]------------ <4> [254.196712] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.196713] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.196787] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.196858] efi_pstore nfnetlink autofs4 <4> [254.196863] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.196866] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.196867] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.196869] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.196874] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.196944] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.196946] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.196948] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.196949] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.196951] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.196952] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.196953] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.196955] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.196956] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.196958] CR2: 0000773620d5c000 CR3: 0000000146ed5006 CR4: 0000000000f72ef0 <4> [254.196959] PKRU: 55555554 <4> [254.196960] Call Trace: <4> [254.196961] <TASK> <4> [254.196965] ? lock_sync+0x100/0x100 <4> [254.196970] ? lock_release+0xd0/0x2b0 <4> [254.196976] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.196981] process_one_work+0x239/0x760 <4> [254.196988] worker_thread+0x200/0x3f0 <4> [254.196990] ? __pfx_worker_thread+0x10/0x10 <4> [254.196993] kthread+0x10d/0x150 <4> [254.196996] ? __pfx_kthread+0x10/0x10 <4> [254.196999] ret_from_fork+0x3d4/0x480 <4> [254.197002] ? __pfx_kthread+0x10/0x10 <4> [254.197005] ret_from_fork_asm+0x1a/0x30 <4> [254.197012] </TASK> <4> [254.197013] irq event stamp: 280481 <4> [254.197015] hardirqs last enabled at (280487): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.197017] hardirqs last disabled at (280492): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.197020] softirqs last enabled at (279472): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.197023] softirqs last disabled at (279465): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.197025] ---[ end trace 0000000000000000 ]--- <5> [254.197605] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39723, lrc_seqno=39723, guc_id=0, flags=0x73 in no process [-1] <7> [254.197614] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.197612] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.197722] ------------[ cut here ]------------ <4> [254.197724] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <7> [254.197719] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [254.197726] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:50/3229 <4> [254.197851] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.197944] efi_pstore nfnetlink autofs4 <4> [254.197951] CPU: 12 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.197955] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.197956] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.197959] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.197966] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.198083] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.198085] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.198089] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.198091] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.198094] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.198096] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.198098] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.198101] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [254.198103] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.198106] CR2: 0000773610096048 CR3: 0000000127fb6002 CR4: 0000000000f72ef0 <4> [254.198108] PKRU: 55555554 <4> [254.198110] Call Trace: <4> [254.198112] <TASK> <4> [254.198119] ? lock_sync+0x100/0x100 <4> [254.198127] ? lock_release+0xd0/0x2b0 <4> [254.198136] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.198146] process_one_work+0x239/0x760 <4> [254.198156] worker_thread+0x200/0x3f0 <4> [254.198161] ? __pfx_worker_thread+0x10/0x10 <4> [254.198165] kthread+0x10d/0x150 <4> [254.198169] ? __pfx_kthread+0x10/0x10 <4> [254.198175] ret_from_fork+0x3d4/0x480 <4> [254.198178] ? __pfx_kthread+0x10/0x10 <4> [254.198183] ret_from_fork_asm+0x1a/0x30 <4> [254.198196] </TASK> <4> [254.198198] irq event stamp: 427287 <4> [254.198200] hardirqs last enabled at (427293): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.198204] hardirqs last disabled at (427298): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.198208] softirqs last enabled at (426344): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [254.198212] softirqs last disabled at (426342): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [254.198215] ---[ end trace 0000000000000000 ]--- <6> [254.198218] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.198333] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.198342] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.198393] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.198645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.198781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.198931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.199066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.199199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.199329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.199462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.199597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.199726] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.199876] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.200043] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.201394] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.211835] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.212146] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.213325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.213447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.213564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.213684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.213803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.213930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.214050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.214169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.214287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.214406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.214525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.214644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.214763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.214886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.215005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.215125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.215243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.215362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.215482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.215606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.215729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.215857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.215982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.216104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.216227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.216349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.216471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.216597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.216723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.216853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.216978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.217103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.217234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.217364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.217494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.217622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.217748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.217878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.218003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.218125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.218251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.218375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.218499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.218622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.218748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.218878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.219002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.219124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.219248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.219370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.219492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.219618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.219741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.219869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.219992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.220116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.220238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.220359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.220485] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.220646] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.220653] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39723, lrc_seqno=39723, guc_id=0, flags=0x73 in no process [-1] <7> [254.220657] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.220758] ------------[ cut here ]------------ <4> [254.220759] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.220761] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:50/3229 <4> [254.220881] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.220973] efi_pstore nfnetlink autofs4 <4> [254.220979] CPU: 12 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.220983] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.220985] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.220987] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.220994] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.221106] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.221108] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.221111] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.221113] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.221115] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.221117] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.221118] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.221120] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [254.221123] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.221125] CR2: 0000773610096048 CR3: 0000000127fb6002 CR4: 0000000000f72ef0 <4> [254.221127] PKRU: 55555554 <4> [254.221128] Call Trace: <4> [254.221130] <TASK> <4> [254.221136] ? lock_sync+0x100/0x100 <4> [254.221143] ? lock_release+0xd0/0x2b0 <4> [254.221152] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.221161] process_one_work+0x239/0x760 <4> [254.221171] worker_thread+0x200/0x3f0 <4> [254.221175] ? __pfx_worker_thread+0x10/0x10 <4> [254.221178] kthread+0x10d/0x150 <4> [254.221182] ? __pfx_kthread+0x10/0x10 <4> [254.221188] ret_from_fork+0x3d4/0x480 <4> [254.221190] ? __pfx_kthread+0x10/0x10 <4> [254.221195] ret_from_fork_asm+0x1a/0x30 <4> [254.221207] </TASK> <4> [254.221209] irq event stamp: 430421 <4> [254.221210] hardirqs last enabled at (430427): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.221214] hardirqs last disabled at (430432): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.221217] softirqs last enabled at (429676): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.221221] softirqs last disabled at (429665): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.221224] ---[ end trace 0000000000000000 ]--- <6> [254.221237] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.221677] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.221686] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.222941] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.223109] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.223264] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.223473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.223563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.223659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.223748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.223837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.223922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.224007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.224094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.224174] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.224268] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.224384] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.225390] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.235830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.236078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.237082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.237161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.237237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.237315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.237391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.237467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.237542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.237616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.237691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.237766] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.237844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.237920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.237994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.238068] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.238141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.238215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.238288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.238361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.238434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.238516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.238598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.238677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.238759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.238846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.238926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.239005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.239081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.239160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.239242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.239320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.239399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.239476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.239561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.239645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.239729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.239809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.239900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.239980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.240062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.240140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.240220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.240297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.240374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.240449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.240528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.240604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.240682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.240762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.240847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.240925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.241001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.241081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.241157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.241237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.241315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.241394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.241472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.241548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.241629] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.241742] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.241746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39723, lrc_seqno=39723, guc_id=0, flags=0x73 in no process [-1] <7> [254.241748] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.241808] ------------[ cut here ]------------ <4> [254.241809] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.241810] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.241897] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.241962] efi_pstore nfnetlink autofs4 <4> [254.241967] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.241970] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.241971] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.241973] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.241978] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.242047] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.242049] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.242051] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.242053] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.242054] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.242055] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.242056] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.242058] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.242059] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.242060] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.242062] PKRU: 55555554 <4> [254.242063] Call Trace: <4> [254.242064] <TASK> <4> [254.242068] ? lock_sync+0x100/0x100 <4> [254.242072] ? lock_release+0xd0/0x2b0 <4> [254.242078] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.242083] process_one_work+0x239/0x760 <4> [254.242089] worker_thread+0x200/0x3f0 <4> [254.242092] ? __pfx_worker_thread+0x10/0x10 <4> [254.242094] kthread+0x10d/0x150 <4> [254.242097] ? __pfx_kthread+0x10/0x10 <4> [254.242101] ret_from_fork+0x3d4/0x480 <4> [254.242103] ? __pfx_kthread+0x10/0x10 <4> [254.242106] ret_from_fork_asm+0x1a/0x30 <4> [254.242114] </TASK> <4> [254.242115] irq event stamp: 285955 <4> [254.242116] hardirqs last enabled at (285961): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.242119] hardirqs last disabled at (285966): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.242121] softirqs last enabled at (285212): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.242124] softirqs last disabled at (285201): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.242126] ---[ end trace 0000000000000000 ]--- <5> [254.242720] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39778, lrc_seqno=39778, guc_id=0, flags=0x73 in no process [-1] <7> [254.242734] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.242902] ------------[ cut here ]------------ <4> [254.242904] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.242906] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:31/3209 <4> [254.243020] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.243123] efi_pstore nfnetlink autofs4 <4> [254.243130] CPU: 12 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.243135] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.243136] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.243140] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.243150] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.243261] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.243263] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.243267] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.243269] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.243271] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.243273] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.243275] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.243277] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [254.243279] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.243282] CR2: 0000773610096048 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [254.243284] PKRU: 55555554 <4> [254.243285] Call Trace: <4> [254.243287] <TASK> <4> [254.243295] ? lock_sync+0x100/0x100 <4> [254.243303] ? lock_release+0xd0/0x2b0 <4> [254.243313] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.243322] process_one_work+0x239/0x760 <4> [254.243333] worker_thread+0x200/0x3f0 <4> [254.243337] ? __pfx_worker_thread+0x10/0x10 <4> [254.243341] kthread+0x10d/0x150 <4> [254.243345] ? __pfx_kthread+0x10/0x10 <4> [254.243350] ret_from_fork+0x3d4/0x480 <4> [254.243353] ? __pfx_kthread+0x10/0x10 <4> [254.243358] ret_from_fork_asm+0x1a/0x30 <4> [254.243371] </TASK> <4> [254.243373] irq event stamp: 204881 <4> [254.243374] hardirqs last enabled at (204887): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.243379] hardirqs last disabled at (204892): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.243382] softirqs last enabled at (204120): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.243387] softirqs last disabled at (204115): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.243390] ---[ end trace 0000000000000000 ]--- <6> [254.243392] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.243508] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.243525] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.243615] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.243721] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.244279] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.244487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.244578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.244671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.244759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.244852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.244939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.245029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.245119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.245202] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.245297] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.245413] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.246420] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.256829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.257079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.258067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.258146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.258222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.258300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.258376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.258454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.258529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.258604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.258679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.258753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.258832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.258907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.258979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.259054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.259130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.259204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.259277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.259351] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.259424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.259507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.259587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.259666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.259749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.259829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.259907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.259983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.260058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.260137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.260219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.260301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.260382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.260461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.260544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.260627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.260711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.260791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.260878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.260956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.261032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.261106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.261185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.261265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.261344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.261422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.261503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.261581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.261658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.261734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.261811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.261900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.261977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.262055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.262131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.262209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.262287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.262366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.262440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.262515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.262593] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.262704] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.262708] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39778, lrc_seqno=39778, guc_id=0, flags=0x73 in no process [-1] <7> [254.262711] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.262770] ------------[ cut here ]------------ <4> [254.262771] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.262772] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.262847] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.262913] efi_pstore nfnetlink autofs4 <4> [254.262918] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.262921] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.262922] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.262923] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.262929] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.263000] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.263002] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.263004] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.263006] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.263007] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.263008] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.263009] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.263011] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.263012] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.263013] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.263015] PKRU: 55555554 <4> [254.263016] Call Trace: <4> [254.263017] <TASK> <4> [254.263021] ? lock_sync+0x100/0x100 <4> [254.263025] ? lock_release+0xd0/0x2b0 <4> [254.263031] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.263037] process_one_work+0x239/0x760 <4> [254.263043] worker_thread+0x200/0x3f0 <4> [254.263045] ? __pfx_worker_thread+0x10/0x10 <4> [254.263048] kthread+0x10d/0x150 <4> [254.263050] ? __pfx_kthread+0x10/0x10 <4> [254.263054] ret_from_fork+0x3d4/0x480 <4> [254.263056] ? __pfx_kthread+0x10/0x10 <4> [254.263060] ret_from_fork_asm+0x1a/0x30 <4> [254.263067] </TASK> <4> [254.263068] irq event stamp: 291315 <4> [254.263069] hardirqs last enabled at (291321): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.263072] hardirqs last disabled at (291326): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.263074] softirqs last enabled at (290412): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.263077] softirqs last disabled at (290407): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.263079] ---[ end trace 0000000000000000 ]--- <6> [254.263081] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.263153] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.263159] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.263564] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.263766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.263861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.263954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.264042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.264128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.264216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.264305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.264392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.264475] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.264569] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.264684] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.265690] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.276411] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.276654] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.277678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.277751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.277819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.277909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.277986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.278063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.278141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.278218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.278294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.278369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.278443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.278517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.278591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.278665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.278737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.278811] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.278898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.278972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.279046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.279127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.279208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.279288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.279370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.279452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.279531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.279609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.279685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.279766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.279853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.279931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.280010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.280088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.280176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.280263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.280348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.280429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.280507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.280584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.280662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.280738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.280816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.280906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.280984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.281064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.281146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.281224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.281302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.281378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.281455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.281529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.281604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.281682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.281758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.281836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.281914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.281996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.282074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.282152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.282232] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.282340] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.282344] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39778, lrc_seqno=39778, guc_id=0, flags=0x73 in no process [-1] <7> [254.282346] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.282405] ------------[ cut here ]------------ <4> [254.282406] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.282407] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.282478] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.282540] efi_pstore nfnetlink autofs4 <4> [254.282544] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.282547] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.282548] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.282550] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.282555] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.282624] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.282625] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.282628] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.282629] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.282630] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.282631] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.282632] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.282634] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.282635] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.282637] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.282638] PKRU: 55555554 <4> [254.282639] Call Trace: <4> [254.282640] <TASK> <4> [254.282644] ? lock_sync+0x100/0x100 <4> [254.282648] ? lock_release+0xd0/0x2b0 <4> [254.282654] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.282659] process_one_work+0x239/0x760 <4> [254.282665] worker_thread+0x200/0x3f0 <4> [254.282668] ? __pfx_worker_thread+0x10/0x10 <4> [254.282671] kthread+0x10d/0x150 <4> [254.282673] ? __pfx_kthread+0x10/0x10 <4> [254.282677] ret_from_fork+0x3d4/0x480 <4> [254.282679] ? __pfx_kthread+0x10/0x10 <4> [254.282683] ret_from_fork_asm+0x1a/0x30 <4> [254.282690] </TASK> <4> [254.282691] irq event stamp: 294417 <4> [254.282692] hardirqs last enabled at (294423): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.282695] hardirqs last disabled at (294428): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.282697] softirqs last enabled at (293352): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.282700] softirqs last disabled at (293343): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.282702] ---[ end trace 0000000000000000 ]--- <7> [254.284057] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.284162] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [254.284584] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39833, lrc_seqno=39833, guc_id=0, flags=0x73 in no process [-1] <7> [254.284586] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.284646] ------------[ cut here ]------------ <4> [254.284647] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.284648] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.284720] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.284782] efi_pstore nfnetlink autofs4 <4> [254.284786] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.284788] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.284789] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.284790] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.284795] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.284877] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.284879] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.284882] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.284883] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.284885] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.284886] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.284887] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.284888] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.284890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.284891] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.284893] PKRU: 55555554 <4> [254.284894] Call Trace: <4> [254.284895] <TASK> <4> [254.284899] ? lock_sync+0x100/0x100 <4> [254.284903] ? lock_release+0xd0/0x2b0 <4> [254.284908] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.284914] process_one_work+0x239/0x760 <4> [254.284920] worker_thread+0x200/0x3f0 <4> [254.284923] ? __pfx_worker_thread+0x10/0x10 <4> [254.284925] kthread+0x10d/0x150 <4> [254.284928] ? __pfx_kthread+0x10/0x10 <4> [254.284932] ret_from_fork+0x3d4/0x480 <4> [254.284934] ? __pfx_kthread+0x10/0x10 <4> [254.284938] ret_from_fork_asm+0x1a/0x30 <4> [254.284945] </TASK> <4> [254.284946] irq event stamp: 295997 <4> [254.284947] hardirqs last enabled at (296003): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.284950] hardirqs last disabled at (296008): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.284952] softirqs last enabled at (295836): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.284955] softirqs last disabled at (295823): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.284957] ---[ end trace 0000000000000000 ]--- <6> [254.284959] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.285028] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.285035] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.285173] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.285374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.285461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.285554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.285643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.285730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.285815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.285915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.286002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.286084] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.286180] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.286294] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.287313] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.297829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.298080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.299106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.299179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.299248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.299317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.299386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.299454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.299522] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.299592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.299661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.299730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.299797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.299877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.299954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.300022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.300090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.300156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.300223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.300290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.300356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.300430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.300505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.300577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.300649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.300719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.300788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.300873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.300949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.301022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.301098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.301170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.301242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.301314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.301390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.301466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.301542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.301615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.301686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.301757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.301834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.301913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.301987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.302062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.302134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.302205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.302278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.302350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.302420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.302489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.302559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.302626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.302694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.302765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.302837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.302918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.302988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.303059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.303127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.303197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.303268] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.303370] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.303375] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39833, lrc_seqno=39833, guc_id=0, flags=0x73 in no process [-1] <7> [254.303377] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.303431] ------------[ cut here ]------------ <4> [254.303432] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.303433] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.303498] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.303554] efi_pstore nfnetlink autofs4 <4> [254.303559] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.303562] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.303563] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.303564] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.303569] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.303633] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.303635] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.303637] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.303638] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.303640] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.303641] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.303643] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.303644] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.303646] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.303647] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.303649] PKRU: 55555554 <4> [254.303650] Call Trace: <4> [254.303652] <TASK> <4> [254.303655] ? lock_sync+0x100/0x100 <4> [254.303660] ? lock_release+0xd0/0x2b0 <4> [254.303665] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.303670] process_one_work+0x239/0x760 <4> [254.303676] worker_thread+0x200/0x3f0 <4> [254.303679] ? __pfx_worker_thread+0x10/0x10 <4> [254.303681] kthread+0x10d/0x150 <4> [254.303684] ? __pfx_kthread+0x10/0x10 <4> [254.303687] ret_from_fork+0x3d4/0x480 <4> [254.303689] ? __pfx_kthread+0x10/0x10 <4> [254.303693] ret_from_fork_asm+0x1a/0x30 <4> [254.303699] </TASK> <4> [254.303701] irq event stamp: 299127 <4> [254.303702] hardirqs last enabled at (299133): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.303705] hardirqs last disabled at (299138): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.303707] softirqs last enabled at (298150): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.303709] softirqs last disabled at (298143): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.303711] ---[ end trace 0000000000000000 ]--- <6> [254.303713] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.303777] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.303782] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.304839] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.304941] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.305037] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.305233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.305314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.305399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.305481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.305560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.305638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.305716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.305795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.305877] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.305964] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.306073] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.307068] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.316829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.317070] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.318300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.318376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.318447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.318518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.318587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.318657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.318726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.318795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.318878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.318949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.319018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.319086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.319154] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.319222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.319289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.319356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.319423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.319489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.319556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.319632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.319706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.319777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.319858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.319930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.320000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.320070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.320138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.320210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.320281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.320351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.320421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.320492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.320569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.320645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.320720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.320793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.320873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.320948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.321024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.321092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.321163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.321233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.321302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.321371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.321447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.321520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.321591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.321661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.321731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.321800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.321881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.321960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.322042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.322114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.322183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.322253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.322322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.322390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.322462] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.322563] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.322567] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39833, lrc_seqno=39833, guc_id=0, flags=0x73 in no process [-1] <7> [254.322570] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.322623] ------------[ cut here ]------------ <4> [254.322624] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.322626] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.322690] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.322753] efi_pstore nfnetlink autofs4 <4> [254.322758] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.322760] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.322762] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.322763] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.322768] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.322834] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.322835] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.322837] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.322839] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.322840] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.322841] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.322842] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.322843] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.322845] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.322846] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.322848] PKRU: 55555554 <4> [254.322849] Call Trace: <4> [254.322850] <TASK> <4> [254.322854] ? lock_sync+0x100/0x100 <4> [254.322858] ? lock_release+0xd0/0x2b0 <4> [254.322863] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.322869] process_one_work+0x239/0x760 <4> [254.322875] worker_thread+0x200/0x3f0 <4> [254.322878] ? __pfx_worker_thread+0x10/0x10 <4> [254.322881] kthread+0x10d/0x150 <4> [254.322884] ? __pfx_kthread+0x10/0x10 <4> [254.322887] ret_from_fork+0x3d4/0x480 <4> [254.322890] ? __pfx_kthread+0x10/0x10 <4> [254.322893] ret_from_fork_asm+0x1a/0x30 <4> [254.322900] </TASK> <4> [254.322901] irq event stamp: 302243 <4> [254.322903] hardirqs last enabled at (302249): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.322905] hardirqs last disabled at (302254): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.322908] softirqs last enabled at (301346): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.322910] softirqs last disabled at (301335): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.322913] ---[ end trace 0000000000000000 ]--- <5> [254.323499] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39888, lrc_seqno=39888, guc_id=0, flags=0x73 in no process [-1] <7> [254.323502] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.323564] ------------[ cut here ]------------ <4> [254.323565] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.323566] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.323637] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.323702] efi_pstore nfnetlink autofs4 <4> [254.323707] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.323710] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.323712] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.323713] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.323718] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.323788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.323790] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.323793] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.323794] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.323796] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.323797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.323799] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.323800] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.323802] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.323803] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.323805] PKRU: 55555554 <4> [254.323806] Call Trace: <4> [254.323807] <TASK> <4> [254.323811] ? lock_sync+0x100/0x100 <4> [254.323816] ? lock_release+0xd0/0x2b0 <4> [254.323825] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.323831] process_one_work+0x239/0x760 <4> [254.323837] worker_thread+0x200/0x3f0 <4> [254.323840] ? __pfx_worker_thread+0x10/0x10 <4> [254.323842] kthread+0x10d/0x150 <4> [254.323845] ? __pfx_kthread+0x10/0x10 <4> [254.323849] ret_from_fork+0x3d4/0x480 <4> [254.323852] ? __pfx_kthread+0x10/0x10 <4> [254.323855] ret_from_fork_asm+0x1a/0x30 <4> [254.323863] </TASK> <4> [254.323864] irq event stamp: 304967 <4> [254.323865] hardirqs last enabled at (304973): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.323868] hardirqs last disabled at (304978): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.323871] softirqs last enabled at (301346): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.323873] softirqs last disabled at (301335): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.323876] ---[ end trace 0000000000000000 ]--- <6> [254.324863] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.324950] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.324957] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.325010] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.325114] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.325220] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.325422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.325512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.325607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.325694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.325781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.325877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.325962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.326048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.326129] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.326223] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.326339] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.327342] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.337828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.338076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.339031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.339104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.339174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.339244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.339314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.339383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.339452] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.339523] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.339593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.339662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.339731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.339800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.339884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.339959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.340032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.340102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.340171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.340241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.340309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.340384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.340458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.340529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.340601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.340671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.340741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.340811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.340900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.340980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.341060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.341139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.341218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.341295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.341379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.341463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.341545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.341625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.341707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.341787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.341873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.341950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.342029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.342107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.342186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.342267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.342350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.342428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.342506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.342583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.342661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.342737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.342813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.342903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.342981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.343056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.343131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.343207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.343281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.343355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.343434] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.343545] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.343549] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39888, lrc_seqno=39888, guc_id=0, flags=0x73 in no process [-1] <7> [254.343552] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.343611] ------------[ cut here ]------------ <4> [254.343612] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.343613] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.343685] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.343763] efi_pstore nfnetlink autofs4 <4> [254.343767] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.343770] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.343772] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.343773] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.343778] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.343854] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.343856] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.343859] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.343860] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.343861] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.343863] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.343864] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.343865] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.343867] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.343869] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.343870] PKRU: 55555554 <4> [254.343872] Call Trace: <4> [254.343873] <TASK> <4> [254.343876] ? lock_sync+0x100/0x100 <4> [254.343881] ? lock_release+0xd0/0x2b0 <4> [254.343887] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.343892] process_one_work+0x239/0x760 <4> [254.343899] worker_thread+0x200/0x3f0 <4> [254.343901] ? __pfx_worker_thread+0x10/0x10 <4> [254.343904] kthread+0x10d/0x150 <4> [254.343907] ? __pfx_kthread+0x10/0x10 <4> [254.343911] ret_from_fork+0x3d4/0x480 <4> [254.343913] ? __pfx_kthread+0x10/0x10 <4> [254.343917] ret_from_fork_asm+0x1a/0x30 <4> [254.343924] </TASK> <4> [254.343925] irq event stamp: 308071 <4> [254.343926] hardirqs last enabled at (308077): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.343929] hardirqs last disabled at (308082): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.343931] softirqs last enabled at (306992): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.343934] softirqs last disabled at (306985): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.343937] ---[ end trace 0000000000000000 ]--- <6> [254.344872] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.344949] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.344956] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.345009] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.345111] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.345217] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.345418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.345505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.345596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.345682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.345768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.345859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.345943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.346028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.346109] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.346202] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.346319] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.347325] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.357828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.358074] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.359081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.359160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.359236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.359313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.359389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.359464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.359539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.359614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.359691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.359768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.359850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.359925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.360000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.360075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.360150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.360224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.360299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.360374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.360448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.360529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.360610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.360687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.360767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.360848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.360928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.361007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.361085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.361164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.361244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.361322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.361401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.361479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.361562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.361645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.361728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.361807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.361897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.361973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.362052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.362126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.362205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.362283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.362358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.362433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.362511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.362587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.362664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.362743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.362827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.362903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.362979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.363059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.363135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.363212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.363290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.363371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.363449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.363525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.363605] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.363713] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.363717] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39888, lrc_seqno=39888, guc_id=0, flags=0x73 in no process [-1] <7> [254.363719] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.363778] ------------[ cut here ]------------ <4> [254.363779] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.363781] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.363863] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.363927] efi_pstore nfnetlink autofs4 <4> [254.363932] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.363934] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.363936] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.363937] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.363942] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.364011] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.364013] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.364015] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.364016] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.364017] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.364019] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.364020] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.364021] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.364023] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.364024] CR2: 00007d00b023a8d0 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.364025] PKRU: 55555554 <4> [254.364027] Call Trace: <4> [254.364028] <TASK> <4> [254.364031] ? lock_sync+0x100/0x100 <4> [254.364036] ? lock_release+0xd0/0x2b0 <4> [254.364041] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.364047] process_one_work+0x239/0x760 <4> [254.364053] worker_thread+0x200/0x3f0 <4> [254.364056] ? __pfx_worker_thread+0x10/0x10 <4> [254.364058] kthread+0x10d/0x150 <4> [254.364061] ? __pfx_kthread+0x10/0x10 <4> [254.364065] ret_from_fork+0x3d4/0x480 <4> [254.364067] ? __pfx_kthread+0x10/0x10 <4> [254.364070] ret_from_fork_asm+0x1a/0x30 <4> [254.364077] </TASK> <4> [254.364078] irq event stamp: 311163 <4> [254.364079] hardirqs last enabled at (311169): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.364082] hardirqs last disabled at (311174): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.364084] softirqs last enabled at (310420): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.364087] softirqs last disabled at (310409): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.364089] ---[ end trace 0000000000000000 ]--- <5> [254.364675] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39943, lrc_seqno=39943, guc_id=0, flags=0x73 in no process [-1] <7> [254.364680] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.364912] ------------[ cut here ]------------ <4> [254.364915] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.364918] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:31/3209 <4> [254.365067] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.365175] efi_pstore nfnetlink autofs4 <4> [254.365184] CPU: 12 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.365188] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.365190] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.365192] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.365202] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.365357] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.365360] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.365364] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.365366] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.365368] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.365370] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.365372] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.365374] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [254.365376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.365378] CR2: 0000773610096048 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [254.365380] PKRU: 55555554 <4> [254.365382] Call Trace: <4> [254.365384] <TASK> <4> [254.365390] ? lock_sync+0x100/0x100 <4> [254.365399] ? lock_release+0xd0/0x2b0 <4> [254.365408] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.365419] process_one_work+0x239/0x760 <4> [254.365430] worker_thread+0x200/0x3f0 <4> [254.365434] ? __pfx_worker_thread+0x10/0x10 <4> [254.365437] kthread+0x10d/0x150 <4> [254.365442] ? __pfx_kthread+0x10/0x10 <4> [254.365447] ret_from_fork+0x3d4/0x480 <4> [254.365450] ? __pfx_kthread+0x10/0x10 <4> [254.365455] ret_from_fork_asm+0x1a/0x30 <4> [254.365467] </TASK> <4> [254.365469] irq event stamp: 211599 <4> [254.365470] hardirqs last enabled at (211605): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.365474] hardirqs last disabled at (211610): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.365477] softirqs last enabled at (210838): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.365480] softirqs last disabled at (210767): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.365483] ---[ end trace 0000000000000000 ]--- <6> [254.365486] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.365600] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <7> [254.365562] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <6> [254.365613] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.365670] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.365928] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.366139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.366233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.366328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.366418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.366507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.366596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.366687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.366775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.366866] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.366961] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.367078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.368090] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.377829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.378081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.379061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.379141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.379217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.379293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.379368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.379442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.379517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.379591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.379665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.379739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.379813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.379904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.379980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.380055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.380131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.380207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.380283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.380357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.380431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.380511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.380593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.380672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.380754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.380838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.380917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.380992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.381070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.381150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.381229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.381307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.381389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.381471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.381557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.381640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.381724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.381805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.381893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.381972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.382048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.382122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.382199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.382275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.382352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.382426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.382506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.382585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.382664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.382741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.382818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.382907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.382982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.383061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.383137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.383213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.383309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.383401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.383476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.383551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.383629] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.384840] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.384846] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39943, lrc_seqno=39943, guc_id=0, flags=0x73 in no process [-1] <7> [254.384849] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.384932] ------------[ cut here ]------------ <4> [254.384934] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.384935] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.385010] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof <7> [254.384991] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [254.385043] kvm_intel <4> [254.385045] usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 <4> [254.385080] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.385083] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.385084] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.385085] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.385090] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <7> [254.385100] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [254.385161] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.385163] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.385165] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.385166] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.385167] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.385168] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.385169] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.385170] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.385172] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.385173] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.385174] PKRU: 55555554 <4> [254.385175] Call Trace: <4> [254.385176] <TASK> <4> [254.385179] ? lock_sync+0x100/0x100 <4> [254.385184] ? lock_release+0xd0/0x2b0 <4> [254.385189] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.385195] process_one_work+0x239/0x760 <4> [254.385202] worker_thread+0x200/0x3f0 <4> [254.385205] ? __pfx_worker_thread+0x10/0x10 <4> [254.385207] kthread+0x10d/0x150 <4> [254.385210] ? __pfx_kthread+0x10/0x10 <4> [254.385214] ret_from_fork+0x3d4/0x480 <4> [254.385216] ? __pfx_kthread+0x10/0x10 <4> [254.385219] ret_from_fork_asm+0x1a/0x30 <4> [254.385227] </TASK> <4> [254.385228] irq event stamp: 214723 <4> [254.385229] hardirqs last enabled at (214729): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.385232] hardirqs last disabled at (214734): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.385234] softirqs last enabled at (213814): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.385237] softirqs last disabled at (213807): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.385240] ---[ end trace 0000000000000000 ]--- <6> [254.385242] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.385312] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.385319] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.385726] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.385929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.386019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.386112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.386203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.386291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.386378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.386464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.386554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.386638] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.386732] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.386857] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.387866] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.397829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.398080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.399102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.399182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.399258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.399334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.399409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.399483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.399558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.399631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.399706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.399780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.399868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.399942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.400017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.400089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.400163] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.400235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.400309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.400384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.400459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.400540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.400620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.400699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.400778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.400863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.400940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.401019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.401098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.401178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.401259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.401336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.401414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.401491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.401574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.401657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.401739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.401824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.401908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.401986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.402063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.402139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.402218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.402294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.402370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.402448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.402530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.402608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.402687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.402763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.402847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.402923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.403000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.403084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.403163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.403241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.403318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.403394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.403470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.403544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.403626] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.403736] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.403741] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39943, lrc_seqno=39943, guc_id=0, flags=0x73 in no process [-1] <7> [254.403743] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.403802] ------------[ cut here ]------------ <4> [254.403803] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.403804] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.403888] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.403953] efi_pstore nfnetlink autofs4 <4> [254.403957] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.403960] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.403961] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.403963] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.403968] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.404036] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.404037] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.404040] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.404041] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.404042] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.404043] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.404045] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.404046] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.404048] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.404049] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.404050] PKRU: 55555554 <4> [254.404051] Call Trace: <4> [254.404053] <TASK> <4> [254.404056] ? lock_sync+0x100/0x100 <4> [254.404061] ? lock_release+0xd0/0x2b0 <4> [254.404066] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.404072] process_one_work+0x239/0x760 <4> [254.404078] worker_thread+0x200/0x3f0 <4> [254.404081] ? __pfx_worker_thread+0x10/0x10 <4> [254.404083] kthread+0x10d/0x150 <4> [254.404086] ? __pfx_kthread+0x10/0x10 <4> [254.404090] ret_from_fork+0x3d4/0x480 <4> [254.404092] ? __pfx_kthread+0x10/0x10 <4> [254.404095] ret_from_fork_asm+0x1a/0x30 <4> [254.404103] </TASK> <4> [254.404104] irq event stamp: 217811 <4> [254.404105] hardirqs last enabled at (217817): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.404108] hardirqs last disabled at (217822): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.404110] softirqs last enabled at (217068): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.404112] softirqs last disabled at (217063): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.404115] ---[ end trace 0000000000000000 ]--- <5> [254.404696] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39998, lrc_seqno=39998, guc_id=0, flags=0x73 in no process [-1] <7> [254.404699] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.404758] ------------[ cut here ]------------ <4> [254.404759] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.404761] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.404838] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.404900] efi_pstore nfnetlink autofs4 <4> [254.404905] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.404907] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.404908] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.404910] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.404914] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.404983] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.404984] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.404986] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.404987] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.404989] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.404990] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.404991] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.404992] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.404994] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.404995] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.404996] PKRU: 55555554 <4> [254.404998] Call Trace: <4> [254.404999] <TASK> <4> [254.405002] ? lock_sync+0x100/0x100 <4> [254.405006] ? lock_release+0xd0/0x2b0 <4> [254.405012] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.405017] process_one_work+0x239/0x760 <4> [254.405023] worker_thread+0x200/0x3f0 <4> [254.405026] ? __pfx_worker_thread+0x10/0x10 <4> [254.405028] kthread+0x10d/0x150 <4> [254.405031] ? __pfx_kthread+0x10/0x10 <4> [254.405034] ret_from_fork+0x3d4/0x480 <4> [254.405036] ? __pfx_kthread+0x10/0x10 <4> [254.405040] ret_from_fork_asm+0x1a/0x30 <4> [254.405047] </TASK> <4> [254.405048] irq event stamp: 220499 <4> [254.405049] hardirqs last enabled at (220505): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.405051] hardirqs last disabled at (220510): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.405054] softirqs last enabled at (217068): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.405056] softirqs last disabled at (217063): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.405058] ---[ end trace 0000000000000000 ]--- <6> [254.405925] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.406005] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.406013] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.406067] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.406171] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.406277] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.406481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.406569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.406662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.406750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.406847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.406937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.407023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.407110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.407191] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.407285] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.407402] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.408413] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.418828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.419081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.420101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.420181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.420257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.420334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.420411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.420488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.420564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.420641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.420718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.420793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.420881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.420955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.421030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.421104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.421178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.421251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.421323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.421396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.421472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.421555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.421637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.421717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.421795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.421880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.421958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.422034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.422110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.422191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.422272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.422350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.422429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.422510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.422597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.422682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.422766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.422854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.422933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.423010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.423087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.423165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.423247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.423326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.423403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.423478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.423557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.423632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.423708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.423783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.423872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.423948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.424023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.424104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.424183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.424262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.424338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.424415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.424490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.424565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.424647] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.424759] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.424763] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39998, lrc_seqno=39998, guc_id=0, flags=0x73 in no process [-1] <7> [254.424766] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.426329] ------------[ cut here ]------------ <4> [254.426330] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.426332] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.426411] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.426476] efi_pstore nfnetlink autofs4 <4> [254.426481] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.426484] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.426485] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.426487] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.426493] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.426563] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.426565] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.426567] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.426569] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.426570] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.426571] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.426572] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.426574] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.426575] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.426577] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.426578] PKRU: 55555554 <4> [254.426579] Call Trace: <4> [254.426580] <TASK> <4> [254.426584] ? lock_sync+0x100/0x100 <4> [254.426589] ? lock_release+0xd0/0x2b0 <4> [254.426595] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.426601] process_one_work+0x239/0x760 <4> [254.426607] worker_thread+0x200/0x3f0 <4> [254.426610] ? __pfx_worker_thread+0x10/0x10 <4> [254.426612] kthread+0x10d/0x150 <4> [254.426615] ? __pfx_kthread+0x10/0x10 <4> [254.426619] ret_from_fork+0x3d4/0x480 <4> [254.426621] ? __pfx_kthread+0x10/0x10 <4> [254.426625] ret_from_fork_asm+0x1a/0x30 <4> [254.426632] </TASK> <4> [254.426633] irq event stamp: 223601 <4> [254.426634] hardirqs last enabled at (223607): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.426637] hardirqs last disabled at (223612): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.426639] softirqs last enabled at (222836): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.426642] softirqs last disabled at (222829): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.426644] ---[ end trace 0000000000000000 ]--- <6> [254.426646] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.426716] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.426723] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.427470] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.427574] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.427681] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.427889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.427978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.428068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.428155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.428240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.428324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.428408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.428493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.428574] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.428668] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.428785] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.429805] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.439829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.440080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.441104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.441183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.441259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.441337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.441413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.441490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.441565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.441639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.441713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.441788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.441919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.441993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.442070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.442145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.442219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.442294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.442368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.442444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.442518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.442599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.442679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.442758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.442867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.442945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.443022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.443099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.443175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.443255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.443334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.443412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.443489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.443565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.443648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.443733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.443823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.443906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.443986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.444063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.444138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.444212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.444288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.444364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.444441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.444515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.444596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.444676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.444754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.444838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.444917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.444992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.445068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.445146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.445222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.445299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.445377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.445457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.445534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.445611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.445690] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.445801] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.445805] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39998, lrc_seqno=39998, guc_id=0, flags=0x73 in no process [-1] <7> [254.445807] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.446735] ------------[ cut here ]------------ <4> [254.446736] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.446737] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.446818] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.446895] efi_pstore nfnetlink autofs4 <4> [254.446900] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.446903] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.446904] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.446905] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.446911] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.446981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.446983] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.446985] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.446986] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.446988] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.446989] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.446990] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.446991] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.446993] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.446994] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.446996] PKRU: 55555554 <4> [254.446997] Call Trace: <4> [254.446998] <TASK> <4> [254.447002] ? lock_sync+0x100/0x100 <4> [254.447007] ? lock_release+0xd0/0x2b0 <4> [254.447012] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.447018] process_one_work+0x239/0x760 <4> [254.447025] worker_thread+0x200/0x3f0 <4> [254.447027] ? __pfx_worker_thread+0x10/0x10 <4> [254.447030] kthread+0x10d/0x150 <4> [254.447033] ? __pfx_kthread+0x10/0x10 <4> [254.447036] ret_from_fork+0x3d4/0x480 <4> [254.447039] ? __pfx_kthread+0x10/0x10 <4> [254.447042] ret_from_fork_asm+0x1a/0x30 <4> [254.447049] </TASK> <4> [254.447050] irq event stamp: 227117 <4> [254.447051] hardirqs last enabled at (227123): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.447054] hardirqs last disabled at (227128): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.447056] softirqs last enabled at (226386): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.447059] softirqs last disabled at (226379): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.447061] ---[ end trace 0000000000000000 ]--- <5> [254.447666] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40053, lrc_seqno=40053, guc_id=0, flags=0x73 in no process [-1] <7> [254.447671] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.447673] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.447781] ------------[ cut here ]------------ <4> [254.447783] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <7> [254.447778] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [254.447785] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:25/3203 <4> [254.447953] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.448046] efi_pstore nfnetlink autofs4 <4> [254.448052] CPU: 12 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.448057] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.448058] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.448060] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.448068] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.448182] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.448185] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.448188] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.448190] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.448192] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.448194] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.448196] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.448198] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [254.448200] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.448202] CR2: 0000773610096048 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [254.448204] PKRU: 55555554 <4> [254.448206] Call Trace: <4> [254.448208] <TASK> <4> [254.448214] ? lock_sync+0x100/0x100 <4> [254.448222] ? lock_release+0xd0/0x2b0 <4> [254.448230] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.448240] process_one_work+0x239/0x760 <4> [254.448250] worker_thread+0x200/0x3f0 <4> [254.448254] ? __pfx_worker_thread+0x10/0x10 <4> [254.448258] kthread+0x10d/0x150 <4> [254.448262] ? __pfx_kthread+0x10/0x10 <4> [254.448267] ret_from_fork+0x3d4/0x480 <4> [254.448270] ? __pfx_kthread+0x10/0x10 <4> [254.448275] ret_from_fork_asm+0x1a/0x30 <4> [254.448286] </TASK> <4> [254.448288] irq event stamp: 317139 <4> [254.448290] hardirqs last enabled at (317145): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.448294] hardirqs last disabled at (317150): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.448297] softirqs last enabled at (316396): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.448300] softirqs last disabled at (316201): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.448303] ---[ end trace 0000000000000000 ]--- <6> [254.448306] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.448420] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.448430] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.448618] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.448885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.448977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.449069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.449159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.449247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.449333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.449421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.449507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.449588] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.449681] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.449796] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.450814] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.460830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.461083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.462081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.462160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.462236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.462313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.462389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.462464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.462541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.462618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.462693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.462767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.462849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.462924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.462998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.463074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.463151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.463226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.463300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.463375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.463449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.463531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.463613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.463694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.463776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.463860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.463938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.464015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.464094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.464175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.464255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.464334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.464412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.464490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.464572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.464655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.464737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.464817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.464947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.465028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.465109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.465186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.465264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.465342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.465417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.465494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.465577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.465653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.465729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.465804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.465891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.465965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.466039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.466117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.466192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.466266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.466340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.466422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.466502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.466580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.466660] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.466773] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.466777] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40053, lrc_seqno=40053, guc_id=0, flags=0x73 in no process [-1] <7> [254.466779] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.467691] ------------[ cut here ]------------ <4> [254.467692] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.467694] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:25/3203 <4> [254.467776] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.467843] efi_pstore nfnetlink autofs4 <4> [254.467848] CPU: 0 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.467851] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.467852] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.467853] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.467859] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.467929] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.467931] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.467933] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.467935] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.467936] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.467937] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.467938] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.467940] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.467941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.467943] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.467944] PKRU: 55555554 <4> [254.467945] Call Trace: <4> [254.467946] <TASK> <4> [254.467950] ? lock_sync+0x100/0x100 <4> [254.467955] ? lock_release+0xd0/0x2b0 <4> [254.467960] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.467966] process_one_work+0x239/0x760 <4> [254.467972] worker_thread+0x200/0x3f0 <4> [254.467974] ? __pfx_worker_thread+0x10/0x10 <4> [254.467977] kthread+0x10d/0x150 <4> [254.467980] ? __pfx_kthread+0x10/0x10 <4> [254.467983] ret_from_fork+0x3d4/0x480 <4> [254.467985] ? __pfx_kthread+0x10/0x10 <4> [254.467989] ret_from_fork_asm+0x1a/0x30 <4> [254.467996] </TASK> <4> [254.467997] irq event stamp: 320429 <4> [254.467998] hardirqs last enabled at (320435): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.468001] hardirqs last disabled at (320440): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.468003] softirqs last enabled at (319512): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.468006] softirqs last disabled at (319505): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.468008] ---[ end trace 0000000000000000 ]--- <7> [254.468020] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.468121] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [254.468217] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.468287] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.468293] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.468830] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.469038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.469131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.469226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.469315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.469402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.469486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.469571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.469656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.469737] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.469835] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.469952] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.470972] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.480829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.481079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.482107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.482186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.482262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.482341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.482417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.482491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.482565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.482639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.482713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.482786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.482875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.482949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.483022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.483096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.483173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.483250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.483327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.483402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.483476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.483556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.483637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.483714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.483793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.483879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.483959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.484039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.484117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.484197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.484277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.484356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.484436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.484517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.484603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.484687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.484772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.484859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.484939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.485016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.485096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.485173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.485252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.485330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.485406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.485482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.485561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.485638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.485719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.485798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.485884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.485960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.486037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.486121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.486201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.486280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.486358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.486434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.486510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.486586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.486665] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.486775] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.486779] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40053, lrc_seqno=40053, guc_id=0, flags=0x73 in no process [-1] <7> [254.486782] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.486851] ------------[ cut here ]------------ <4> [254.486852] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.486854] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.486926] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.486989] efi_pstore nfnetlink autofs4 <4> [254.486993] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.486996] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.486997] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.486999] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.487004] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.487074] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.487076] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.487078] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.487079] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.487081] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.487082] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.487083] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.487084] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.487086] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.487087] CR2: 00007d00b023a8d0 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.487089] PKRU: 55555554 <4> [254.487090] Call Trace: <4> [254.487091] <TASK> <4> [254.487095] ? lock_sync+0x100/0x100 <4> [254.487099] ? lock_release+0xd0/0x2b0 <4> [254.487105] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.487110] process_one_work+0x239/0x760 <4> [254.487117] worker_thread+0x200/0x3f0 <4> [254.487119] ? __pfx_worker_thread+0x10/0x10 <4> [254.487122] kthread+0x10d/0x150 <4> [254.487125] ? __pfx_kthread+0x10/0x10 <4> [254.487129] ret_from_fork+0x3d4/0x480 <4> [254.487131] ? __pfx_kthread+0x10/0x10 <4> [254.487135] ret_from_fork_asm+0x1a/0x30 <4> [254.487142] </TASK> <4> [254.487144] irq event stamp: 441041 <4> [254.487145] hardirqs last enabled at (441047): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.487148] hardirqs last disabled at (441052): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.487151] softirqs last enabled at (439944): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.487154] softirqs last disabled at (439935): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.487156] ---[ end trace 0000000000000000 ]--- <5> [254.487895] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40108, lrc_seqno=40108, guc_id=0, flags=0x73 in no process [-1] <7> [254.487901] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.488008] ------------[ cut here ]------------ <4> [254.488011] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.488013] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.488092] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.488176] efi_pstore nfnetlink autofs4 <4> [254.488182] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.488185] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.488187] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.488189] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.488196] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.488282] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.488285] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.488288] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.488291] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.488293] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.488294] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.488296] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.488297] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.488300] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.488301] CR2: 0000773620d60000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.488303] PKRU: 55555554 <4> [254.488305] Call Trace: <4> [254.488308] <TASK> <4> [254.488313] ? lock_sync+0x100/0x100 <4> [254.488319] ? lock_release+0xd0/0x2b0 <4> [254.488324] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.488331] process_one_work+0x239/0x760 <4> [254.488338] worker_thread+0x200/0x3f0 <4> [254.488340] ? __pfx_worker_thread+0x10/0x10 <4> [254.488343] kthread+0x10d/0x150 <4> [254.488348] ? __pfx_kthread+0x10/0x10 <4> [254.488353] ret_from_fork+0x3d4/0x480 <4> [254.488356] ? __pfx_kthread+0x10/0x10 <4> [254.488360] ret_from_fork_asm+0x1a/0x30 <4> [254.488368] </TASK> <4> [254.488369] irq event stamp: 232903 <4> [254.488371] hardirqs last enabled at (232909): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.488374] hardirqs last disabled at (232914): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.488376] softirqs last enabled at (229934): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [254.488379] softirqs last disabled at (229932): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [254.488381] ---[ end trace 0000000000000000 ]--- <6> [254.488383] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.488462] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.488469] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.488483] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.488587] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.488693] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.488912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.489001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.489094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.489184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.489274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.489360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.489446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.489533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.489614] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.489708] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.489830] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.490845] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.500829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.501079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.502099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.502179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.502254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.502331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.502406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.502481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.502555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.502630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.502704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.502778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.502884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.502961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.503039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.503114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.503188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.503263] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.503338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.503412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.503487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.503567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.503647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.503727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.503807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.503897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.503978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.504058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.504136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.504216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.504295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.504372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.504451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.504528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.504612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.504694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.504777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.504866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.504945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.505023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.505102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.505180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.505260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.505338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.505415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.505492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.505575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.505655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.505734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.505810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.505904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.505981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.506056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.506135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.506211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.506292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.506372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.506450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.506526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.506601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.506680] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.506790] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.506795] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40108, lrc_seqno=40108, guc_id=0, flags=0x73 in no process [-1] <7> [254.506797] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.507722] ------------[ cut here ]------------ <4> [254.507723] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.507724] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.507806] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.507875] efi_pstore nfnetlink autofs4 <4> [254.507879] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.507882] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.507883] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.507884] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.507889] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.507958] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.507960] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.507962] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.507964] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.507965] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.507966] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.507968] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.507969] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.507971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.507972] CR2: 0000773620d60000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.507973] PKRU: 55555554 <4> [254.507975] Call Trace: <4> [254.507976] <TASK> <4> [254.507979] ? lock_sync+0x100/0x100 <4> [254.507984] ? lock_release+0xd0/0x2b0 <4> [254.507989] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.507995] process_one_work+0x239/0x760 <4> [254.508001] worker_thread+0x200/0x3f0 <4> [254.508004] ? __pfx_worker_thread+0x10/0x10 <4> [254.508006] kthread+0x10d/0x150 <4> [254.508009] ? __pfx_kthread+0x10/0x10 <4> [254.508013] ret_from_fork+0x3d4/0x480 <4> [254.508015] ? __pfx_kthread+0x10/0x10 <4> [254.508019] ret_from_fork_asm+0x1a/0x30 <4> [254.508026] </TASK> <4> [254.508027] irq event stamp: 236149 <4> [254.508028] hardirqs last enabled at (236155): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.508031] hardirqs last disabled at (236160): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.508034] softirqs last enabled at (235232): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.508037] softirqs last disabled at (235225): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.508039] ---[ end trace 0000000000000000 ]--- <7> [254.508052] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.508156] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [254.508254] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.508327] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.508333] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.508740] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.508942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.509029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.509120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.509209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.509296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.509381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.509465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.509551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.509632] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.509726] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.509847] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.510860] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.520828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.521078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.522108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.522189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.522265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.522340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.522415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.522491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.522565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.522639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.522713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.522787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.522877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.522951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.523024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.523098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.523173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.523249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.523323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.523397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.523470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.523551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.523631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.523708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.523786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.523876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.523953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.524030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.524108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.524187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.524271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.524352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.524433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.524512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.524596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.524680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.524762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.524846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.524925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.525005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.525083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.525159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.525239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.525316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.525393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.525468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.525547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.525623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.525700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.525780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.525875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.525953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.526030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.526110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.526187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.526265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.526341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.526417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.526492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.526567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.526645] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.526754] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.526759] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40108, lrc_seqno=40108, guc_id=0, flags=0x73 in no process [-1] <7> [254.526761] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.526823] ------------[ cut here ]------------ <4> [254.526825] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.526826] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.526898] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.526961] efi_pstore nfnetlink autofs4 <4> [254.526965] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.526968] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.526969] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.526971] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.526976] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.527045] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.527047] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.527049] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.527051] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.527052] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.527054] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.527055] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.527057] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.527059] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.527060] CR2: 0000773620d60000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.527062] PKRU: 55555554 <4> [254.527063] Call Trace: <4> [254.527064] <TASK> <4> [254.527068] ? lock_sync+0x100/0x100 <4> [254.527073] ? lock_release+0xd0/0x2b0 <4> [254.527079] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.527084] process_one_work+0x239/0x760 <4> [254.527091] worker_thread+0x200/0x3f0 <4> [254.527094] ? __pfx_worker_thread+0x10/0x10 <4> [254.527096] kthread+0x10d/0x150 <4> [254.527099] ? __pfx_kthread+0x10/0x10 <4> [254.527103] ret_from_fork+0x3d4/0x480 <4> [254.527106] ? __pfx_kthread+0x10/0x10 <4> [254.527109] ret_from_fork_asm+0x1a/0x30 <4> [254.527116] </TASK> <4> [254.527118] irq event stamp: 239245 <4> [254.527119] hardirqs last enabled at (239251): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.527122] hardirqs last disabled at (239256): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.527124] softirqs last enabled at (238336): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.527127] softirqs last disabled at (238325): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.527130] ---[ end trace 0000000000000000 ]--- <5> [254.527720] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40163, lrc_seqno=40163, guc_id=0, flags=0x73 in no process [-1] <7> [254.527723] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.527964] ------------[ cut here ]------------ <4> [254.527967] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.527970] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.528053] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.528157] efi_pstore nfnetlink autofs4 <4> [254.528164] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.528168] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.528171] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.528174] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.528183] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.528267] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.528269] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.528272] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.528274] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.528276] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.528278] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.528280] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.528281] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.528283] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.528284] CR2: 0000773620d63000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.528286] PKRU: 55555554 <4> [254.528288] Call Trace: <4> [254.528290] <TASK> <4> [254.528294] ? lock_sync+0x100/0x100 <4> [254.528300] ? lock_release+0xd0/0x2b0 <4> [254.528306] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.528312] process_one_work+0x239/0x760 <4> [254.528320] worker_thread+0x200/0x3f0 <4> [254.528323] ? __pfx_worker_thread+0x10/0x10 <4> [254.528326] kthread+0x10d/0x150 <4> [254.528330] ? __pfx_kthread+0x10/0x10 <4> [254.528335] ret_from_fork+0x3d4/0x480 <4> [254.528337] ? __pfx_kthread+0x10/0x10 <4> [254.528341] ret_from_fork_asm+0x1a/0x30 <4> [254.528350] </TASK> <4> [254.528352] irq event stamp: 443933 <4> [254.528353] hardirqs last enabled at (443939): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.528357] hardirqs last disabled at (443944): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.528360] softirqs last enabled at (443172): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.528365] softirqs last disabled at (443161): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.528367] ---[ end trace 0000000000000000 ]--- <6> [254.528370] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.528457] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.528474] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.528720] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.528832] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.528938] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.529141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.529231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.529324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.529413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.529500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.529589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.529676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.529763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.529849] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.529944] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.530060] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.531068] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.540828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.541076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.542064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.542143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.542220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.542295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.542372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.542447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.542521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.542595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.542673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.542749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.542836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.542911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.542985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.543058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.543131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.543204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.543279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.543354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.543430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.543512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.543592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.543670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.543749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.543831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.543910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.543987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.544063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.544142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.544220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.544297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.544374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.544451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.544533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.544614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.544695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.544777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.544866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.544944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.545020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.545095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.545173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.545250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.545326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.545401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.545482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.545562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.545641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.545718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.545797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.545879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.545954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.546032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.546106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.546181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.546256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.546332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.546407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.546484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.546566] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.546677] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.546681] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40163, lrc_seqno=40163, guc_id=0, flags=0x73 in no process [-1] <7> [254.546684] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.546741] ------------[ cut here ]------------ <4> [254.546743] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.546744] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.546813] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.546881] efi_pstore nfnetlink autofs4 <4> [254.546885] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.546888] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.546889] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.546890] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.546895] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.546965] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.546966] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.546968] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.546970] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.546971] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.546972] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.546974] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.546975] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.546977] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.546978] CR2: 0000773620d63000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.546980] PKRU: 55555554 <4> [254.546981] Call Trace: <4> [254.546983] <TASK> <4> [254.546987] ? lock_sync+0x100/0x100 <4> [254.546991] ? lock_release+0xd0/0x2b0 <4> [254.546997] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.547003] process_one_work+0x239/0x760 <4> [254.547009] worker_thread+0x200/0x3f0 <4> [254.547012] ? __pfx_worker_thread+0x10/0x10 <4> [254.547014] kthread+0x10d/0x150 <4> [254.547017] ? __pfx_kthread+0x10/0x10 <4> [254.547021] ret_from_fork+0x3d4/0x480 <4> [254.547023] ? __pfx_kthread+0x10/0x10 <4> [254.547027] ret_from_fork_asm+0x1a/0x30 <4> [254.547034] </TASK> <4> [254.547036] irq event stamp: 447029 <4> [254.547037] hardirqs last enabled at (447035): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.547040] hardirqs last disabled at (447040): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.547042] softirqs last enabled at (445932): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.547045] softirqs last disabled at (445923): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.547048] ---[ end trace 0000000000000000 ]--- <6> [254.547050] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.547121] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.547127] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.547533] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.547735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.547824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.547915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.548002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.548089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.548173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.548258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.548343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.548423] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.548517] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.548632] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.549636] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.559827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.560076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.561072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.561151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.561228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.561305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.561381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.561456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.561531] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.561604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.561677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.561752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.561830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.561906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.561979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.562051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.562126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.562202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.562278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.562352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.562426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.562506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.562586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.562664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.562743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.562824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.562901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.562978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.563055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.563134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.563214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.563292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.563371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.563451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.563538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.563625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.563711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.563792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.563880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.563957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.564035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.564110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.564189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.564266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.564344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.564424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.564507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.564585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.564662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.564736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.564812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.564900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.564975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.565053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.565129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.565205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.565280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.565357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.565431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.565507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.565586] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.565695] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.565699] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40163, lrc_seqno=40163, guc_id=0, flags=0x73 in no process [-1] <7> [254.565702] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.565760] ------------[ cut here ]------------ <4> [254.565762] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.565763] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.565836] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.565900] efi_pstore nfnetlink autofs4 <4> [254.565904] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.565907] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.565908] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.565910] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.565914] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.565984] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.565986] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.565988] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.565990] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.565991] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.565992] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.565993] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.565994] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.565996] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.565997] CR2: 0000773620d63000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.565999] PKRU: 55555554 <4> [254.566000] Call Trace: <4> [254.566001] <TASK> <4> [254.566004] ? lock_sync+0x100/0x100 <4> [254.566009] ? lock_release+0xd0/0x2b0 <4> [254.566014] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.566020] process_one_work+0x239/0x760 <4> [254.566025] worker_thread+0x200/0x3f0 <4> [254.566028] ? __pfx_worker_thread+0x10/0x10 <4> [254.566030] kthread+0x10d/0x150 <4> [254.566033] ? __pfx_kthread+0x10/0x10 <4> [254.566037] ret_from_fork+0x3d4/0x480 <4> [254.566039] ? __pfx_kthread+0x10/0x10 <4> [254.566042] ret_from_fork_asm+0x1a/0x30 <4> [254.566049] </TASK> <4> [254.566050] irq event stamp: 450103 <4> [254.566052] hardirqs last enabled at (450109): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.566054] hardirqs last disabled at (450114): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.566056] softirqs last enabled at (448906): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.566059] softirqs last disabled at (448901): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.566061] ---[ end trace 0000000000000000 ]--- <5> [254.567594] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40218, lrc_seqno=40218, guc_id=0, flags=0x73 in no process [-1] <7> [254.567601] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [254.567601] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.567710] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [254.567931] ------------[ cut here ]------------ <4> [254.567934] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.567936] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#13: kworker/u64:25/3203 <4> [254.568056] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.568148] efi_pstore nfnetlink autofs4 <4> [254.568155] CPU: 13 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.568159] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.568161] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.568163] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.568170] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.568283] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.568286] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.568289] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.568291] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.568293] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.568295] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.568297] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.568299] FS: 0000000000000000(0000) GS:ffff8888db317000(0000) knlGS:0000000000000000 <4> [254.568301] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.568303] CR2: 0000773620d5d000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.568305] PKRU: 55555554 <4> [254.568307] Call Trace: <4> [254.568309] <TASK> <4> [254.568315] ? lock_sync+0x100/0x100 <4> [254.568322] ? lock_release+0xd0/0x2b0 <4> [254.568331] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.568340] process_one_work+0x239/0x760 <4> [254.568350] worker_thread+0x200/0x3f0 <4> [254.568354] ? __pfx_worker_thread+0x10/0x10 <4> [254.568358] kthread+0x10d/0x150 <4> [254.568362] ? __pfx_kthread+0x10/0x10 <4> [254.568367] ret_from_fork+0x3d4/0x480 <4> [254.568370] ? __pfx_kthread+0x10/0x10 <4> [254.568375] ret_from_fork_asm+0x1a/0x30 <4> [254.568387] </TASK> <4> [254.568388] irq event stamp: 325663 <4> [254.568390] hardirqs last enabled at (325669): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.568394] hardirqs last disabled at (325674): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.568397] softirqs last enabled at (324906): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.568400] softirqs last disabled at (324849): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.568403] ---[ end trace 0000000000000000 ]--- <6> [254.568406] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.568520] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.568530] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.568580] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.568840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.568979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.569118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.569251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.569383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.569513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.569644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.569778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.569915] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.570058] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.570224] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.571590] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.581874] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.582187] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.583356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.583476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.583594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.583713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.583839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.583960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.584079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.584198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.584317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.584436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.584554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.584673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.584791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.584924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.585045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.585165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.585285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.585404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.585522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.585647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.585771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.585902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.586030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.586152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.586273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.586396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.586517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.586643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.586769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.586897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.587024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.587149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.587280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.587410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.587539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.587666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.587793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.587972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.588078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.588168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.588258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.588343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.588427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.588510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.588596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.588679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.588762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.588854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.588937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.589017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.589098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.589182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.589264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.589351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.589436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.589522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.589603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.589685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.589770] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.589894] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.589899] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40218, lrc_seqno=40218, guc_id=0, flags=0x73 in no process [-1] <7> [254.589901] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.589963] ------------[ cut here ]------------ <4> [254.589965] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.589966] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:25/3203 <4> [254.590042] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.590106] efi_pstore nfnetlink autofs4 <4> [254.590111] CPU: 2 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.590114] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.590115] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.590117] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.590122] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.590196] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.590198] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.590200] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.590202] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.590203] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.590204] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.590206] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.590207] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.590208] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.590210] CR2: 00007e95d846f008 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.590211] PKRU: 55555554 <4> [254.590213] Call Trace: <4> [254.590214] <TASK> <4> [254.590218] ? lock_sync+0x100/0x100 <4> [254.590223] ? lock_release+0xd0/0x2b0 <4> [254.590228] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.590234] process_one_work+0x239/0x760 <4> [254.590240] worker_thread+0x200/0x3f0 <4> [254.590243] ? __pfx_worker_thread+0x10/0x10 <4> [254.590246] kthread+0x10d/0x150 <4> [254.590249] ? __pfx_kthread+0x10/0x10 <4> [254.590252] ret_from_fork+0x3d4/0x480 <4> [254.590254] ? __pfx_kthread+0x10/0x10 <4> [254.590258] ret_from_fork_asm+0x1a/0x30 <4> [254.590265] </TASK> <4> [254.590266] irq event stamp: 328805 <4> [254.590268] hardirqs last enabled at (328811): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.590271] hardirqs last disabled at (328816): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.590273] softirqs last enabled at (327970): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.590275] softirqs last disabled at (327965): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.590278] ---[ end trace 0000000000000000 ]--- <6> [254.590279] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.590356] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.590362] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.591603] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.591737] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.591865] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.592072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.592162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.592257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.592345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.592432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.592520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.592608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.592694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.592776] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.592880] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.592996] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.594006] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.604727] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.604978] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.606028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.606108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.606185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.606262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.606337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.606412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.606487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.606561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.606635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.606709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.606785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.606871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.606947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.607020] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.607095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.607168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.607241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.607313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.607386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.607467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.607549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.607629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.607707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.607784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.607868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.607943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.608017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.608094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.608171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.608251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.608330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.608410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.608496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.608579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.608662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.608741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.608825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.608916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.608999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.609076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.609157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.609237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.609314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.609391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.609471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.609547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.609625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.609704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.609786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.609875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.609952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.610032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.610109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.610186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.610262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.610339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.610415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.610491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.610567] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.610677] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.610682] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40218, lrc_seqno=40218, guc_id=0, flags=0x73 in no process [-1] <7> [254.610684] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.610742] ------------[ cut here ]------------ <4> [254.610743] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.610744] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.610816] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.610884] efi_pstore nfnetlink autofs4 <4> [254.610889] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.610892] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.610893] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.610895] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.610900] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.610970] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.610972] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.610974] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.610976] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.610977] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.610978] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.610979] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.610981] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.610982] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.610984] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.610985] PKRU: 55555554 <4> [254.610986] Call Trace: <4> [254.610987] <TASK> <4> [254.610991] ? lock_sync+0x100/0x100 <4> [254.610996] ? lock_release+0xd0/0x2b0 <4> [254.611001] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.611007] process_one_work+0x239/0x760 <4> [254.611013] worker_thread+0x200/0x3f0 <4> [254.611016] ? __pfx_worker_thread+0x10/0x10 <4> [254.611018] kthread+0x10d/0x150 <4> [254.611021] ? __pfx_kthread+0x10/0x10 <4> [254.611025] ret_from_fork+0x3d4/0x480 <4> [254.611027] ? __pfx_kthread+0x10/0x10 <4> [254.611030] ret_from_fork_asm+0x1a/0x30 <4> [254.611037] </TASK> <4> [254.611038] irq event stamp: 245177 <4> [254.611040] hardirqs last enabled at (245183): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.611042] hardirqs last disabled at (245188): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.611044] softirqs last enabled at (244274): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.611047] softirqs last disabled at (244263): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.611049] ---[ end trace 0000000000000000 ]--- <5> [254.611660] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40273, lrc_seqno=40273, guc_id=0, flags=0x73 in no process [-1] <7> [254.611669] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.611775] ------------[ cut here ]------------ <4> [254.611777] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.611779] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.611872] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.611953] efi_pstore nfnetlink autofs4 <4> [254.611958] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.611964] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.611965] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.611967] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.611975] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.612065] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.612067] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.612070] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.612071] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.612073] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.612074] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.612075] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.612077] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.612079] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.612080] CR2: 00007e95d846e008 CR3: 0000000146ed5001 CR4: 0000000000f72ef0 <4> [254.612082] PKRU: 55555554 <4> [254.612083] Call Trace: <4> [254.612084] <TASK> <4> [254.612089] ? lock_sync+0x100/0x100 <4> [254.612095] ? lock_release+0xd0/0x2b0 <4> [254.612101] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.612107] process_one_work+0x239/0x760 <4> [254.612115] worker_thread+0x200/0x3f0 <4> [254.612118] ? __pfx_worker_thread+0x10/0x10 <4> [254.612122] kthread+0x10d/0x150 <4> [254.612125] ? __pfx_kthread+0x10/0x10 <4> [254.612129] ret_from_fork+0x3d4/0x480 <4> [254.612131] ? __pfx_kthread+0x10/0x10 <4> [254.612136] ret_from_fork_asm+0x1a/0x30 <4> [254.612145] </TASK> <4> [254.612146] irq event stamp: 330593 <4> [254.612148] hardirqs last enabled at (330599): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.612152] hardirqs last disabled at (330604): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.612154] softirqs last enabled at (329326): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [254.612157] softirqs last disabled at (329324): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [254.612160] ---[ end trace 0000000000000000 ]--- <6> [254.612163] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.612250] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.612262] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.612452] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.612558] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.612663] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.612875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.612966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.613059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.613148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.613234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.613319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.613405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.613492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.613575] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.613670] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.613790] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.614804] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.624827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.625075] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.626077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.626155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.626231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.626309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.626386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.626462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.626537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.626611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.626684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.626759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.626835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.626910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.626985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.627057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.627129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.627200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.627272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.627345] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.627418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.627498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.627579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.627658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.627738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.627815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.627906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.627986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.628063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.628143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.628222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.628301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.628380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.628458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.628542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.628625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.628708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.628788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.628875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.628951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.629027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.629103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.629179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.629255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.629331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.629405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.629483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.629559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.629635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.629710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.629791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.629876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.629954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.630035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.630112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.630188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.630263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.630341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.630417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.630495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.630577] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.630688] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.630692] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40273, lrc_seqno=40273, guc_id=0, flags=0x73 in no process [-1] <7> [254.630694] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.630753] ------------[ cut here ]------------ <4> [254.630754] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.630755] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.630828] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.630893] efi_pstore nfnetlink autofs4 <4> [254.630897] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.630900] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.630901] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.630903] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.630908] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.630977] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.630979] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.630981] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.630983] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.630984] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.630985] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.630986] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.630988] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.630989] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.630991] CR2: 0000773620d63000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.630992] PKRU: 55555554 <4> [254.630993] Call Trace: <4> [254.630994] <TASK> <4> [254.630998] ? lock_sync+0x100/0x100 <4> [254.631002] ? lock_release+0xd0/0x2b0 <4> [254.631008] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.631013] process_one_work+0x239/0x760 <4> [254.631019] worker_thread+0x200/0x3f0 <4> [254.631022] ? __pfx_worker_thread+0x10/0x10 <4> [254.631024] kthread+0x10d/0x150 <4> [254.631027] ? __pfx_kthread+0x10/0x10 <4> [254.631031] ret_from_fork+0x3d4/0x480 <4> [254.631033] ? __pfx_kthread+0x10/0x10 <4> [254.631036] ret_from_fork_asm+0x1a/0x30 <4> [254.631043] </TASK> <4> [254.631044] irq event stamp: 457867 <4> [254.631045] hardirqs last enabled at (457873): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.631048] hardirqs last disabled at (457878): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.631050] softirqs last enabled at (456776): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.631053] softirqs last disabled at (456767): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.631055] ---[ end trace 0000000000000000 ]--- <6> [254.631991] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.632063] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.632069] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.632083] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.632182] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.632285] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.632489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.632581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.632675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.632762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.632858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.632946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.633031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.633118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.633201] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.633296] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.633413] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.634428] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.644827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.645077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.646099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.646178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.646256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.646334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.646410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.646485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.646560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.646634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.646708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.646783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.646886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.646963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.647038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.647111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.647187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.647261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.647335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.647409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.647487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.647570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.647651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.647729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.647809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.647898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.647976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.648056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.648135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.648214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.648292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.648369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.648448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.648526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.648609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.648692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.648775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.648863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.648943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.649022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.649102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.649180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.649260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.649340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.649418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.649495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.649576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.649652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.649729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.649804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.649891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.649972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.650051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.650132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.650209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.650285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.650361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.650437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.650511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.650586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.650667] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.650779] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.650783] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40273, lrc_seqno=40273, guc_id=0, flags=0x73 in no process [-1] <7> [254.650786] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.651698] ------------[ cut here ]------------ <4> [254.651700] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.651701] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.651775] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.651843] efi_pstore nfnetlink autofs4 <4> [254.651848] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.651851] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.651852] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.651854] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.651859] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.651929] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.651930] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.651933] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.651934] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.651936] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.651937] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.651938] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.651939] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.651941] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.651942] CR2: 00007e95d846e008 CR3: 0000000146ed5001 CR4: 0000000000f72ef0 <4> [254.651944] PKRU: 55555554 <4> [254.651945] Call Trace: <4> [254.651946] <TASK> <4> [254.651950] ? lock_sync+0x100/0x100 <4> [254.651955] ? lock_release+0xd0/0x2b0 <4> [254.651961] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.651967] process_one_work+0x239/0x760 <4> [254.651973] worker_thread+0x200/0x3f0 <4> [254.651976] ? __pfx_worker_thread+0x10/0x10 <4> [254.651979] kthread+0x10d/0x150 <4> [254.651982] ? __pfx_kthread+0x10/0x10 <4> [254.651986] ret_from_fork+0x3d4/0x480 <4> [254.651988] ? __pfx_kthread+0x10/0x10 <4> [254.651991] ret_from_fork_asm+0x1a/0x30 <4> [254.651999] </TASK> <4> [254.652000] irq event stamp: 334459 <4> [254.652002] hardirqs last enabled at (334465): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.652004] hardirqs last disabled at (334470): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.652007] softirqs last enabled at (333356): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.652010] softirqs last disabled at (333347): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.652012] ---[ end trace 0000000000000000 ]--- <5> [254.652594] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40328, lrc_seqno=40328, guc_id=0, flags=0x73 in no process [-1] <7> [254.652597] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [254.652603] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [254.652657] ------------[ cut here ]------------ <4> [254.652658] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.652660] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [254.652733] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp <7> [254.652707] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [254.652756] hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 <4> [254.652800] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.652802] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.652804] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.652805] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.652810] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.652922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.652924] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.652926] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.652928] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.652929] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.652931] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.652932] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.652934] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.652935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.652937] CR2: 0000773620d63000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.652938] PKRU: 55555554 <4> [254.652940] Call Trace: <4> [254.652941] <TASK> <4> [254.652945] ? lock_sync+0x100/0x100 <4> [254.652950] ? lock_release+0xd0/0x2b0 <4> [254.652955] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.652961] process_one_work+0x239/0x760 <4> [254.652967] worker_thread+0x200/0x3f0 <4> [254.652970] ? __pfx_worker_thread+0x10/0x10 <4> [254.652973] kthread+0x10d/0x150 <4> [254.652976] ? __pfx_kthread+0x10/0x10 <4> [254.652980] ret_from_fork+0x3d4/0x480 <4> [254.652982] ? __pfx_kthread+0x10/0x10 <4> [254.652986] ret_from_fork_asm+0x1a/0x30 <4> [254.652993] </TASK> <4> [254.652995] irq event stamp: 460141 <4> [254.652996] hardirqs last enabled at (460147): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.652999] hardirqs last disabled at (460152): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.653001] softirqs last enabled at (459980): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.653004] softirqs last disabled at (459815): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.653006] ---[ end trace 0000000000000000 ]--- <6> [254.653008] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.653080] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.653086] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.653295] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.653498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.653587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.653680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.653767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.653860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.653944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.654028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.654113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.654196] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.654291] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.654409] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.655415] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.665827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.666079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.667100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.667181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.667256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.667332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.667407] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.667482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.667557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.667633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.667708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.667783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.667870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.667948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.668023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.668097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.668172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.668245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.668319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.668393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.668469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.668552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.668633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.668711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.668790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.668876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.668953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.669029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.669106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.669186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.669265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.669345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.669425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.669506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.669591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.669675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.669759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.669841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.669920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.669996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.670072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.670146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.670224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.670301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.670381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.670458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.670540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.670617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.670695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.670771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.670853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.670929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.671004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.671082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.671159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.671239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.671317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.671395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.671470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.671545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.671623] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.671733] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.671737] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40328, lrc_seqno=40328, guc_id=0, flags=0x73 in no process [-1] <7> [254.671739] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.671798] ------------[ cut here ]------------ <4> [254.671799] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.671801] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.671883] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.671948] efi_pstore nfnetlink autofs4 <4> [254.671953] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.671956] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.671957] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.671959] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.671964] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.672036] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.672038] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.672040] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.672042] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.672043] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.672045] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.672046] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.672048] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.672050] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.672051] CR2: 00007e95d846e008 CR3: 0000000146ed5001 CR4: 0000000000f72ef0 <4> [254.672053] PKRU: 55555554 <4> [254.672054] Call Trace: <4> [254.672055] <TASK> <4> [254.672059] ? lock_sync+0x100/0x100 <4> [254.672064] ? lock_release+0xd0/0x2b0 <4> [254.672070] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.672075] process_one_work+0x239/0x760 <4> [254.672082] worker_thread+0x200/0x3f0 <4> [254.672084] ? __pfx_worker_thread+0x10/0x10 <4> [254.672087] kthread+0x10d/0x150 <4> [254.672090] ? __pfx_kthread+0x10/0x10 <4> [254.672094] ret_from_fork+0x3d4/0x480 <4> [254.672096] ? __pfx_kthread+0x10/0x10 <4> [254.672099] ret_from_fork_asm+0x1a/0x30 <4> [254.672107] </TASK> <4> [254.672108] irq event stamp: 339581 <4> [254.672109] hardirqs last enabled at (339587): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.672112] hardirqs last disabled at (339592): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.672115] softirqs last enabled at (338484): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.672117] softirqs last disabled at (338475): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.672120] ---[ end trace 0000000000000000 ]--- <6> [254.673046] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.673119] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.673127] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.673179] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.673282] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.673386] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.673588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.673676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.673770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.673872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.673960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.674044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.674128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.674215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.674296] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.674390] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.674506] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.675512] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.685826] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.686072] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.687018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.687098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.687174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.687251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.687328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.687401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.687475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.687548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.687621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.687694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.687767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.687846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.687920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.687993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.688067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.688140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.688214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.688290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.688366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.688449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.688531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.688610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.688691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.688769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.688851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.688928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.689005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.689083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.689162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.689241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.689320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.689399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.689484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.689572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.689660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.689743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.689829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.689907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.689986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.690066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.690147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.690226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.690303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.690381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.690464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.690542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.690621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.690700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.690780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.690866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.690944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.691025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.691103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.691179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.691255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.691333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.691411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.691487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.691571] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.691686] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.691690] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40328, lrc_seqno=40328, guc_id=0, flags=0x73 in no process [-1] <7> [254.691693] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.691753] ------------[ cut here ]------------ <4> [254.691754] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.691756] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [254.691832] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.691898] efi_pstore nfnetlink autofs4 <4> [254.691902] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.691905] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.691907] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.691908] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.691913] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.691988] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.691990] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.691992] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.691994] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.691995] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.691997] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.691998] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.692000] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.692002] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.692003] CR2: 00007e95d846e008 CR3: 0000000146ed5001 CR4: 0000000000f72ef0 <4> [254.692005] PKRU: 55555554 <4> [254.692006] Call Trace: <4> [254.692008] <TASK> <4> [254.692012] ? lock_sync+0x100/0x100 <4> [254.692017] ? lock_release+0xd0/0x2b0 <4> [254.692022] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.692028] process_one_work+0x239/0x760 <4> [254.692035] worker_thread+0x200/0x3f0 <4> [254.692038] ? __pfx_worker_thread+0x10/0x10 <4> [254.692041] kthread+0x10d/0x150 <4> [254.692044] ? __pfx_kthread+0x10/0x10 <4> [254.692048] ret_from_fork+0x3d4/0x480 <4> [254.692050] ? __pfx_kthread+0x10/0x10 <4> [254.692053] ret_from_fork_asm+0x1a/0x30 <4> [254.692061] </TASK> <4> [254.692062] irq event stamp: 342683 <4> [254.692063] hardirqs last enabled at (342689): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.692066] hardirqs last disabled at (342694): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.692069] softirqs last enabled at (341592): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.692072] softirqs last disabled at (341585): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.692074] ---[ end trace 0000000000000000 ]--- <5> [254.692998] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40383, lrc_seqno=40383, guc_id=0, flags=0x73 in no process [-1] <7> [254.693005] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.693127] ------------[ cut here ]------------ <4> [254.693129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.693130] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.693206] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.693290] efi_pstore nfnetlink autofs4 <4> [254.693297] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.693301] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.693303] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.693305] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.693314] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.693398] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.693401] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.693403] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.693405] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.693407] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.693409] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.693411] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.693412] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.693415] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.693417] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.693419] PKRU: 55555554 <4> [254.693421] Call Trace: <4> [254.693422] <TASK> <4> [254.693428] ? lock_sync+0x100/0x100 <4> [254.693432] ? lock_release+0xd0/0x2b0 <4> [254.693438] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.693444] process_one_work+0x239/0x760 <4> [254.693450] worker_thread+0x200/0x3f0 <4> [254.693453] ? __pfx_worker_thread+0x10/0x10 <4> [254.693456] kthread+0x10d/0x150 <4> [254.693459] ? __pfx_kthread+0x10/0x10 <4> [254.693462] ret_from_fork+0x3d4/0x480 <4> [254.693465] ? __pfx_kthread+0x10/0x10 <4> [254.693468] ret_from_fork_asm+0x1a/0x30 <4> [254.693476] </TASK> <4> [254.693477] irq event stamp: 250381 <4> [254.693478] hardirqs last enabled at (250387): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.693481] hardirqs last disabled at (250392): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.693484] softirqs last enabled at (249232): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.693487] softirqs last disabled at (249085): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.693489] ---[ end trace 0000000000000000 ]--- <6> [254.693491] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.693575] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.693582] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.693595] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.693700] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.693806] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.694028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.694119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.694215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.694302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.694390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.694475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.694560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.694647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.694729] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.694832] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.694952] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.695962] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.705829] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.706092] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.707111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.707197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.707275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.707352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.707428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.707504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.707580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.707656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.707733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.707810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.707906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.707982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.708055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.708130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.708204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.708277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.708351] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.708426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.708503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.708588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.708670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.708749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.708832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.708911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.708987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.709064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.709143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.709225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.709304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.709383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.709462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.709542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.709625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.709708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.709791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.709880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.709959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.710035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.710112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.710186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.710265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.710343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.710423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.710501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.710582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.710661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.710738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.710814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.710911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.710990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.711069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.711148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.711225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.711302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.711378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.711455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.711531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.711607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.711685] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.711799] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.711804] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40383, lrc_seqno=40383, guc_id=0, flags=0x73 in no process [-1] <7> [254.711807] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.711885] ------------[ cut here ]------------ <4> [254.711887] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.711888] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.711960] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.712028] efi_pstore nfnetlink autofs4 <4> [254.712033] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.712036] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.712037] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.712039] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.712044] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.712114] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.712116] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.712118] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.712120] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.712121] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.712122] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.712123] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.712125] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.712126] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.712128] CR2: 0000773620d60000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.712129] PKRU: 55555554 <4> [254.712130] Call Trace: <4> [254.712132] <TASK> <4> [254.712136] ? lock_sync+0x100/0x100 <4> [254.712141] ? lock_release+0xd0/0x2b0 <4> [254.712146] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.712152] process_one_work+0x239/0x760 <4> [254.712159] worker_thread+0x200/0x3f0 <4> [254.712161] ? __pfx_worker_thread+0x10/0x10 <4> [254.712164] kthread+0x10d/0x150 <4> [254.712167] ? __pfx_kthread+0x10/0x10 <4> [254.712170] ret_from_fork+0x3d4/0x480 <4> [254.712173] ? __pfx_kthread+0x10/0x10 <4> [254.712176] ret_from_fork_asm+0x1a/0x30 <4> [254.712183] </TASK> <4> [254.712185] irq event stamp: 253499 <4> [254.712186] hardirqs last enabled at (253505): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.712189] hardirqs last disabled at (253510): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.712191] softirqs last enabled at (252396): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.712194] softirqs last disabled at (252389): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.712196] ---[ end trace 0000000000000000 ]--- <6> [254.712198] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.712269] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.712276] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.712682] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.712892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.712982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.713074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.713161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.713250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.713335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.713423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.713510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.713592] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.713688] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.713806] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.714846] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.724827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.725086] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.726120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.726205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.726280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.726357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.726432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.726507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.726581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.726654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.726727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.726801] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.726890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.726967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.727044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.727119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.727194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.727271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.727349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.727423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.727496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.727576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.727656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.727735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.727814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.727905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.727984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.728064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.728145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.728225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.728305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.728382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.728459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.728536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.728619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.728703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.728785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.728873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.728955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.729035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.729115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.729191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.729272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.729349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.729427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.729502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.729585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.729664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.729743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.729822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.729900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.729975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.730050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.730129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.730205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.730283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.730362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.730440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.730516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.730591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.730669] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.730781] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.730785] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40383, lrc_seqno=40383, guc_id=0, flags=0x73 in no process [-1] <7> [254.730787] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.731711] ------------[ cut here ]------------ <4> [254.731713] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.731714] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.731804] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.731876] efi_pstore nfnetlink autofs4 <4> [254.731881] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.731884] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.731885] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.731886] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.731892] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.731962] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.731963] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.731966] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.731967] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.731968] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.731970] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.731971] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.731972] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.731974] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.731975] CR2: 0000773620d60000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.731977] PKRU: 55555554 <4> [254.731978] Call Trace: <4> [254.731979] <TASK> <4> [254.731983] ? lock_sync+0x100/0x100 <4> [254.731987] ? lock_release+0xd0/0x2b0 <4> [254.731993] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.731998] process_one_work+0x239/0x760 <4> [254.732005] worker_thread+0x200/0x3f0 <4> [254.732008] ? __pfx_worker_thread+0x10/0x10 <4> [254.732010] kthread+0x10d/0x150 <4> [254.732013] ? __pfx_kthread+0x10/0x10 <4> [254.732016] ret_from_fork+0x3d4/0x480 <4> [254.732019] ? __pfx_kthread+0x10/0x10 <4> [254.732022] ret_from_fork_asm+0x1a/0x30 <4> [254.732029] </TASK> <4> [254.732030] irq event stamp: 256593 <4> [254.732032] hardirqs last enabled at (256599): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.732034] hardirqs last disabled at (256604): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.732037] softirqs last enabled at (255962): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.732039] softirqs last disabled at (255953): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.732042] ---[ end trace 0000000000000000 ]--- <7> [254.732055] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.732159] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [254.733009] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40438, lrc_seqno=40438, guc_id=0, flags=0x73 in no process [-1] <7> [254.733014] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.733083] ------------[ cut here ]------------ <4> [254.733084] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.733086] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.733160] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.733229] efi_pstore nfnetlink autofs4 <4> [254.733233] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.733236] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.733237] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.733239] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.733244] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.733316] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.733318] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.733321] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.733323] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.733324] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.733326] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.733327] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.733329] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.733330] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.733332] CR2: 0000773620d66000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.733334] PKRU: 55555554 <4> [254.733335] Call Trace: <4> [254.733336] <TASK> <4> [254.733340] ? lock_sync+0x100/0x100 <4> [254.733345] ? lock_release+0xd0/0x2b0 <4> [254.733351] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.733357] process_one_work+0x239/0x760 <4> [254.733364] worker_thread+0x200/0x3f0 <4> [254.733367] ? __pfx_worker_thread+0x10/0x10 <4> [254.733369] kthread+0x10d/0x150 <4> [254.733373] ? __pfx_kthread+0x10/0x10 <4> [254.733377] ret_from_fork+0x3d4/0x480 <4> [254.733379] ? __pfx_kthread+0x10/0x10 <4> [254.733383] ret_from_fork_asm+0x1a/0x30 <4> [254.733390] </TASK> <4> [254.733392] irq event stamp: 345711 <4> [254.733393] hardirqs last enabled at (345717): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.733396] hardirqs last disabled at (345722): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.733399] softirqs last enabled at (344264): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.733402] softirqs last disabled at (344259): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.733404] ---[ end trace 0000000000000000 ]--- <6> [254.733407] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.733478] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.733487] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.733540] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.733800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.733948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.734090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.734224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.734356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.734487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.734621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.734755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.734890] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.735034] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.735200] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.736583] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <6> [254.738946] xe 0000:03:00.0: [drm] PL2 disabled for channel 0, val 0x00000000 <7> [254.742258] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x2a2a2828 <7> [254.742391] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x2b2b2b2b <7> [254.746831] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.747143] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.748347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.748467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.748586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.748706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.748886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.749006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.749125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.749244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.749363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.749482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.749601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.749720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.749845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.749966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.750085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.750204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.750323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.750442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.750561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.750686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.750811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.750939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.751065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.751188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.751310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.751434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.751556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.751682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.751811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.751941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.752068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.752193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.752324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.752455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.752585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.752713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.752847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.752973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.753097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.753220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.753346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.753471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.753595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.753717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.753849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.753974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.754098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.754220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.754344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.754466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.754589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.754715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.754863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.754988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.755110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.755235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.755357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.755479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.755605] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.755765] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.755772] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40438, lrc_seqno=40438, guc_id=0, flags=0x73 in no process [-1] <7> [254.755776] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.756394] ------------[ cut here ]------------ <4> [254.756396] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.756399] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [254.756482] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.756553] efi_pstore nfnetlink autofs4 <4> [254.756559] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.756563] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.756564] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.756566] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.756572] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.756647] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.756649] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.756652] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.756653] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.756655] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.756656] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.756657] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.756658] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.756660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.756661] CR2: 000000c0004c5000 CR3: 000000012c4bb005 CR4: 0000000000f72ef0 <4> [254.756663] PKRU: 55555554 <4> [254.756664] Call Trace: <4> [254.756665] <TASK> <4> [254.756669] ? lock_sync+0x100/0x100 <4> [254.756674] ? lock_release+0xd0/0x2b0 <4> [254.756680] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.756685] process_one_work+0x239/0x760 <4> [254.756692] worker_thread+0x200/0x3f0 <4> [254.756695] ? __pfx_worker_thread+0x10/0x10 <4> [254.756697] kthread+0x10d/0x150 <4> [254.756700] ? __pfx_kthread+0x10/0x10 <4> [254.756704] ret_from_fork+0x3d4/0x480 <4> [254.756707] ? __pfx_kthread+0x10/0x10 <4> [254.756710] ret_from_fork_asm+0x1a/0x30 <4> [254.756718] </TASK> <4> [254.756720] irq event stamp: 468575 <4> [254.756721] hardirqs last enabled at (468581): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.756724] hardirqs last disabled at (468586): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.756727] softirqs last enabled at (467686): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.756729] softirqs last disabled at (467603): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.756732] ---[ end trace 0000000000000000 ]--- <6> [254.756734] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.756807] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.756817] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.757225] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.757381] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.757535] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.757739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.757832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.757925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.758013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.758100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.758189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.758277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.758365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.758448] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.758543] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.758660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.759669] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.770391] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.770641] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.771688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.771769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.771854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.771931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.772006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.772081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.772156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.772230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.772304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.772378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.772450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.772523] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.772598] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.772671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.772745] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.772823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.772896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.772968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.773041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.773120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.773202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.773285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.773367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.773444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.773521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.773597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.773673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.773750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.773832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.773910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.773989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.774067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.774150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.774233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.774317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.774396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.774475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.774552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.774629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.774705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.774784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.774870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.774947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.775022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.775104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.775184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.775263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.775341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.775419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.775495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.775570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.775647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.775723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.775799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.775884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.775960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.776035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.776110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.776188] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.776302] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.776306] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40438, lrc_seqno=40438, guc_id=0, flags=0x73 in no process [-1] <7> [254.776309] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.776369] ------------[ cut here ]------------ <4> [254.776370] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.776371] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [254.776444] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.776512] efi_pstore nfnetlink autofs4 <4> [254.776516] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.776519] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.776520] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.776522] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.776527] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.776597] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.776599] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.776601] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.776603] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.776604] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.776605] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.776606] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.776608] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.776609] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.776611] CR2: 000000c0004c5000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.776612] PKRU: 55555554 <4> [254.776613] Call Trace: <4> [254.776615] <TASK> <4> [254.776618] ? lock_sync+0x100/0x100 <4> [254.776623] ? lock_release+0xd0/0x2b0 <4> [254.776629] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.776635] process_one_work+0x239/0x760 <4> [254.776641] worker_thread+0x200/0x3f0 <4> [254.776644] ? __pfx_worker_thread+0x10/0x10 <4> [254.776646] kthread+0x10d/0x150 <4> [254.776649] ? __pfx_kthread+0x10/0x10 <4> [254.776653] ret_from_fork+0x3d4/0x480 <4> [254.776655] ? __pfx_kthread+0x10/0x10 <4> [254.776659] ret_from_fork_asm+0x1a/0x30 <4> [254.776666] </TASK> <4> [254.776667] irq event stamp: 471679 <4> [254.776669] hardirqs last enabled at (471685): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.776672] hardirqs last disabled at (471690): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.776675] softirqs last enabled at (470812): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.776678] softirqs last disabled at (470805): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.776680] ---[ end trace 0000000000000000 ]--- <5> [254.777665] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40493, lrc_seqno=40493, guc_id=0, flags=0x73 in no process [-1] <7> [254.777672] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.777772] ------------[ cut here ]------------ <4> [254.777774] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.777776] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.777870] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.777957] efi_pstore nfnetlink autofs4 <4> [254.777962] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.777967] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.777968] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.777971] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.777979] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.778069] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.778072] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.778074] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.778076] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.778078] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.778080] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.778082] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.778085] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.778086] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.778088] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.778090] PKRU: 55555554 <4> [254.778091] Call Trace: <4> [254.778093] <TASK> <4> [254.778098] ? lock_sync+0x100/0x100 <4> [254.778104] ? lock_release+0xd0/0x2b0 <4> [254.778111] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.778118] process_one_work+0x239/0x760 <4> [254.778125] worker_thread+0x200/0x3f0 <4> [254.778129] ? __pfx_worker_thread+0x10/0x10 <4> [254.778131] kthread+0x10d/0x150 <4> [254.778135] ? __pfx_kthread+0x10/0x10 <4> [254.778140] ret_from_fork+0x3d4/0x480 <4> [254.778143] ? __pfx_kthread+0x10/0x10 <4> [254.778147] ret_from_fork_asm+0x1a/0x30 <4> [254.778155] </TASK> <4> [254.778156] irq event stamp: 261311 <4> [254.778158] hardirqs last enabled at (261317): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.778161] hardirqs last disabled at (261322): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.778163] softirqs last enabled at (260568): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.778166] softirqs last disabled at (260561): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.778168] ---[ end trace 0000000000000000 ]--- <6> [254.778170] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.778250] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.778256] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.778310] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.778415] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.778522] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.778727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.778816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.778924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.779015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.779106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.779192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.779279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.779366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.779447] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.779539] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.779653] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.780663] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.790826] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.791079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.792097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.792178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.792254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.792331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.792406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.792480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.792557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.792634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.792710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.792784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.792869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.792944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.793021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.793098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.793173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.793247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.793322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.793396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.793468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.793549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.793628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.793707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.793787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.793872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.793949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.794024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.794101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.794181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.794264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.794345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.794425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.794505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.794587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.794671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.794756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.794836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.794914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.794991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.795067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.795143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.795225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.795304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.795382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.795458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.795539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.795615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.795692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.795767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.795848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.795923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.795998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.796079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.796159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.796238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.796315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.796392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.796467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.796543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.796623] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.796732] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.796736] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40493, lrc_seqno=40493, guc_id=0, flags=0x73 in no process [-1] <7> [254.796739] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.796797] ------------[ cut here ]------------ <4> [254.796799] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.796800] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [254.796887] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.796957] efi_pstore nfnetlink autofs4 <4> [254.796962] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.796965] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.796966] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.796967] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.796972] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.797042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.797044] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.797046] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.797048] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.797049] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.797050] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.797051] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.797053] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.797054] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.797056] CR2: 000000c0004c5000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.797057] PKRU: 55555554 <4> [254.797058] Call Trace: <4> [254.797059] <TASK> <4> [254.797063] ? lock_sync+0x100/0x100 <4> [254.797068] ? lock_release+0xd0/0x2b0 <4> [254.797073] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.797079] process_one_work+0x239/0x760 <4> [254.797085] worker_thread+0x200/0x3f0 <4> [254.797088] ? __pfx_worker_thread+0x10/0x10 <4> [254.797090] kthread+0x10d/0x150 <4> [254.797093] ? __pfx_kthread+0x10/0x10 <4> [254.797096] ret_from_fork+0x3d4/0x480 <4> [254.797098] ? __pfx_kthread+0x10/0x10 <4> [254.797102] ret_from_fork_asm+0x1a/0x30 <4> [254.797109] </TASK> <4> [254.797110] irq event stamp: 476385 <4> [254.797112] hardirqs last enabled at (476391): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.797114] hardirqs last disabled at (476396): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.797117] softirqs last enabled at (475642): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.797119] softirqs last disabled at (475617): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.797122] ---[ end trace 0000000000000000 ]--- <6> [254.797123] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.797193] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.797200] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.797610] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.797812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.797915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.798008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.798094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.798178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.798262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.798347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.798434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.798517] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.798613] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.798746] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.799751] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.810477] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.810727] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.811674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.811752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.811832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.811912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.811991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.812069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.812143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.812216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.812289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.812363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.812435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.812507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.812580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.812653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.812727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.812799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.812881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.812959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.813036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.813118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.813200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.813280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.813361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.813437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.813513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.813588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.813663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.813741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.813822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.813900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.813977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.814056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.814140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.814224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.814311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.814394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.814474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.814551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.814626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.814700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.814777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.814862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.814938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.815013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.815092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.815169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.815245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.815324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.815404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.815480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.815557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.815637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.815713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.815789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.815874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.815954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.816031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.816107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.816186] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.816296] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.816301] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40493, lrc_seqno=40493, guc_id=0, flags=0x73 in no process [-1] <7> [254.816303] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.816361] ------------[ cut here ]------------ <4> [254.816362] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.816364] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [254.816434] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.816497] efi_pstore nfnetlink autofs4 <4> [254.816501] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.816504] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.816505] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.816506] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.816511] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.816580] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.816582] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.816584] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.816586] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.816587] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.816588] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.816589] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.816591] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.816592] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.816594] CR2: 000000c0004c5000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.816595] PKRU: 55555554 <4> [254.816596] Call Trace: <4> [254.816597] <TASK> <4> [254.816601] ? lock_sync+0x100/0x100 <4> [254.816605] ? lock_release+0xd0/0x2b0 <4> [254.816611] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.816616] process_one_work+0x239/0x760 <4> [254.816622] worker_thread+0x200/0x3f0 <4> [254.816625] ? __pfx_worker_thread+0x10/0x10 <4> [254.816627] kthread+0x10d/0x150 <4> [254.816630] ? __pfx_kthread+0x10/0x10 <4> [254.816634] ret_from_fork+0x3d4/0x480 <4> [254.816636] ? __pfx_kthread+0x10/0x10 <4> [254.816639] ret_from_fork_asm+0x1a/0x30 <4> [254.816646] </TASK> <4> [254.816647] irq event stamp: 479493 <4> [254.816648] hardirqs last enabled at (479499): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.816651] hardirqs last disabled at (479504): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.816653] softirqs last enabled at (478626): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.816656] softirqs last disabled at (478619): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.816658] ---[ end trace 0000000000000000 ]--- <7> [254.818018] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.818129] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [254.818227] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40548, lrc_seqno=40548, guc_id=0, flags=0x73 in no process [-1] <7> [254.818230] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.818289] ------------[ cut here ]------------ <4> [254.818291] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.818292] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [254.818364] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.818426] efi_pstore nfnetlink autofs4 <4> [254.818431] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.818434] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.818435] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.818437] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.818442] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.818512] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.818514] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.818516] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.818518] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.818519] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.818521] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.818522] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.818524] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.818526] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.818527] CR2: 000000c0004c5000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.818529] PKRU: 55555554 <4> [254.818530] Call Trace: <4> [254.818532] <TASK> <4> [254.818536] ? lock_sync+0x100/0x100 <4> [254.818540] ? lock_release+0xd0/0x2b0 <4> [254.818546] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.818552] process_one_work+0x239/0x760 <4> [254.818558] worker_thread+0x200/0x3f0 <4> [254.818561] ? __pfx_worker_thread+0x10/0x10 <4> [254.818564] kthread+0x10d/0x150 <4> [254.818567] ? __pfx_kthread+0x10/0x10 <4> [254.818571] ret_from_fork+0x3d4/0x480 <4> [254.818573] ? __pfx_kthread+0x10/0x10 <4> [254.818576] ret_from_fork_asm+0x1a/0x30 <4> [254.818584] </TASK> <4> [254.818585] irq event stamp: 480953 <4> [254.818587] hardirqs last enabled at (480959): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.818589] hardirqs last disabled at (480964): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.818592] softirqs last enabled at (478626): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.818595] softirqs last disabled at (478619): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.818597] ---[ end trace 0000000000000000 ]--- <6> [254.818599] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.818668] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.818674] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.818841] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.819048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.819139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.819233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.819323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.819410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.819494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.819579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.819667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.819749] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.819852] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.819971] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.820981] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.830828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.831080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.832080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.832159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.832237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.832314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.832393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.832471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.832547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.832621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.832696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.832770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.832848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.832921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.832993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.833064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.833137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.833209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.833280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.833353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.833427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.833509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.833591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.833671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.833750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.833830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.833908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.833983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.834059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.834139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.834218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.834296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.834374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.834451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.834533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.834615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.834697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.834778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.834875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.834957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.835035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.835111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.835190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.835266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.835343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.835417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.835495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.835570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.835647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.835720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.835801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.835893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.835971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.836049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.836126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.836203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.836279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.836356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.836431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.836506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.836584] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.836697] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.836701] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40548, lrc_seqno=40548, guc_id=0, flags=0x73 in no process [-1] <7> [254.836703] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.836762] ------------[ cut here ]------------ <4> [254.836763] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.836764] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.836841] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.836906] efi_pstore nfnetlink autofs4 <4> [254.836911] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.836914] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.836915] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.836916] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.836921] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.836990] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.836992] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.836994] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.836996] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.836997] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.836998] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.836999] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.837001] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.837002] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.837004] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.837005] PKRU: 55555554 <4> [254.837006] Call Trace: <4> [254.837007] <TASK> <4> [254.837011] ? lock_sync+0x100/0x100 <4> [254.837016] ? lock_release+0xd0/0x2b0 <4> [254.837021] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.837027] process_one_work+0x239/0x760 <4> [254.837033] worker_thread+0x200/0x3f0 <4> [254.837035] ? __pfx_worker_thread+0x10/0x10 <4> [254.837038] kthread+0x10d/0x150 <4> [254.837041] ? __pfx_kthread+0x10/0x10 <4> [254.837044] ret_from_fork+0x3d4/0x480 <4> [254.837046] ? __pfx_kthread+0x10/0x10 <4> [254.837050] ret_from_fork_asm+0x1a/0x30 <4> [254.837057] </TASK> <4> [254.837058] irq event stamp: 266479 <4> [254.837059] hardirqs last enabled at (266485): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.837062] hardirqs last disabled at (266490): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.837064] softirqs last enabled at (265576): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.837067] softirqs last disabled at (265569): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.837069] ---[ end trace 0000000000000000 ]--- <6> [254.837071] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.837142] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.837149] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.838386] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.838493] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.838599] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.838810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.838938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.839033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.839120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.839205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.839289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.839375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.839463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.839548] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.839642] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.839757] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.840766] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.851497] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.851748] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.852756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.852839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.852916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.852992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.853065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.853137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.853210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.853283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.853356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.853428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.853502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.853575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.853648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.853723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.853799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.853885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.853961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.854035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.854109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.854189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.854271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.854351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.854429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.854505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.854579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.854653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.854728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.854806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.854895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.854976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.855057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.855135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.855218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.855301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.855384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.855464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.855545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.855626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.855706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.855782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.855871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.855949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.856027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.856103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.856182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.856259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.856341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.856419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.856498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.856573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.856649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.856729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.856805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.856892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.856968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.857046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.857120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.857194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.857271] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.857383] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.857387] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40548, lrc_seqno=40548, guc_id=0, flags=0x73 in no process [-1] <7> [254.857389] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.857447] ------------[ cut here ]------------ <4> [254.857448] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.857450] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:50/3229 <4> [254.857522] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.857588] efi_pstore nfnetlink autofs4 <4> [254.857593] CPU: 2 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.857596] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.857597] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.857599] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.857605] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.857676] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.857678] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.857680] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.857682] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.857683] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.857685] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.857686] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.857688] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [254.857690] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.857692] CR2: 000000c0004c5000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.857693] PKRU: 55555554 <4> [254.857695] Call Trace: <4> [254.857696] <TASK> <4> [254.857700] ? lock_sync+0x100/0x100 <4> [254.857705] ? lock_release+0xd0/0x2b0 <4> [254.857711] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.857716] process_one_work+0x239/0x760 <4> [254.857722] worker_thread+0x200/0x3f0 <4> [254.857725] ? __pfx_worker_thread+0x10/0x10 <4> [254.857727] kthread+0x10d/0x150 <4> [254.857730] ? __pfx_kthread+0x10/0x10 <4> [254.857734] ret_from_fork+0x3d4/0x480 <4> [254.857736] ? __pfx_kthread+0x10/0x10 <4> [254.857739] ret_from_fork_asm+0x1a/0x30 <4> [254.857746] </TASK> <4> [254.857748] irq event stamp: 484729 <4> [254.857749] hardirqs last enabled at (484735): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.857751] hardirqs last disabled at (484740): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.857754] softirqs last enabled at (483856): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.857756] softirqs last disabled at (483849): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.857758] ---[ end trace 0000000000000000 ]--- <7> [254.859285] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.859393] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [254.859521] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40603, lrc_seqno=40603, guc_id=0, flags=0x73 in no process [-1] <7> [254.859524] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.859587] ------------[ cut here ]------------ <4> [254.859588] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.859589] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.859663] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.859729] efi_pstore nfnetlink autofs4 <4> [254.859734] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.859738] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.859739] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.859741] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.859746] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.859824] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.859827] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.859831] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.859834] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.859835] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.859837] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.859838] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.859840] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.859842] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.859843] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.859845] PKRU: 55555554 <4> [254.859847] Call Trace: <4> [254.859848] <TASK> <4> [254.859852] ? lock_sync+0x100/0x100 <4> [254.859857] ? lock_release+0xd0/0x2b0 <4> [254.859862] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.859868] process_one_work+0x239/0x760 <4> [254.859875] worker_thread+0x200/0x3f0 <4> [254.859877] ? __pfx_worker_thread+0x10/0x10 <4> [254.859880] kthread+0x10d/0x150 <4> [254.859883] ? __pfx_kthread+0x10/0x10 <4> [254.859887] ret_from_fork+0x3d4/0x480 <4> [254.859889] ? __pfx_kthread+0x10/0x10 <4> [254.859892] ret_from_fork_asm+0x1a/0x30 <4> [254.859900] </TASK> <4> [254.859901] irq event stamp: 268775 <4> [254.859902] hardirqs last enabled at (268781): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.859905] hardirqs last disabled at (268786): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.859907] softirqs last enabled at (268618): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.859910] softirqs last disabled at (268611): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.859912] ---[ end trace 0000000000000000 ]--- <6> [254.859914] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.859986] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.859992] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.860201] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.860406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.860495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.860588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.860676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.860762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.860854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.860944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.861032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.861115] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.861209] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.861325] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.862331] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.872826] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.873077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.874036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.874117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.874194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.874271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.874347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.874423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.874496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.874570] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.874644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.874721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.874798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.874896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.874971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.875045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.875119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.875193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.875266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.875339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.875412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.875492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.875572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.875650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.875731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.875807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.875903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.875979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.876056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.876139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.876223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.876303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.876383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.876466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.876553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.876637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.876721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.876802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.876892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.876970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.877046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.877122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.877201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.877277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.877354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.877429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.877509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.877586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.877662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.877737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.877817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.877913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.877992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.878072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.878149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.878227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.878303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.878380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.878455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.878533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.878616] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.878727] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.878731] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40603, lrc_seqno=40603, guc_id=0, flags=0x73 in no process [-1] <7> [254.878734] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.878792] ------------[ cut here ]------------ <4> [254.878793] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.878794] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.878883] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.878946] efi_pstore nfnetlink autofs4 <4> [254.878951] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.878954] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.878955] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.878956] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.878961] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.879031] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.879032] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.879035] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.879036] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.879037] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.879038] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.879040] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.879041] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.879043] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.879044] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.879045] PKRU: 55555554 <4> [254.879047] Call Trace: <4> [254.879048] <TASK> <4> [254.879051] ? lock_sync+0x100/0x100 <4> [254.879056] ? lock_release+0xd0/0x2b0 <4> [254.879061] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.879067] process_one_work+0x239/0x760 <4> [254.879074] worker_thread+0x200/0x3f0 <4> [254.879076] ? __pfx_worker_thread+0x10/0x10 <4> [254.879079] kthread+0x10d/0x150 <4> [254.879082] ? __pfx_kthread+0x10/0x10 <4> [254.879086] ret_from_fork+0x3d4/0x480 <4> [254.879088] ? __pfx_kthread+0x10/0x10 <4> [254.879092] ret_from_fork_asm+0x1a/0x30 <4> [254.879099] </TASK> <4> [254.879100] irq event stamp: 271907 <4> [254.879101] hardirqs last enabled at (271913): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.879105] hardirqs last disabled at (271918): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.879107] softirqs last enabled at (271164): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.879110] softirqs last disabled at (271153): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.879112] ---[ end trace 0000000000000000 ]--- <6> [254.879114] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.879184] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.879190] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.880432] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.880538] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.880646] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.880873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.880968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.881064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.881152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.881239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.881324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.881409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.881495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.881581] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.881677] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.881793] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.882809] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.892830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.893078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.894078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.894158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.894234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.894312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.894387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.894462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.894537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.894612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.894687] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.894762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.894839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.894914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.894990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.895066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.895142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.895217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.895291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.895365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.895441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.895524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.895607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.895684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.895762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.895843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.895921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.895997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.896074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.896153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.896232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.896313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.896395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.896475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.896560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.896644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.896727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.896808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.896900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.896976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.897056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.897135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.897216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.897293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.897370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.897445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.897525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.897604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.897683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.897759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.897838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.897914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.897987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.898065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.898140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.898216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.898292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.898369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.898442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.898516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.898595] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.898708] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.898712] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40603, lrc_seqno=40603, guc_id=0, flags=0x73 in no process [-1] <7> [254.898715] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.898773] ------------[ cut here ]------------ <4> [254.898774] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.898775] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.898855] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.898922] efi_pstore nfnetlink autofs4 <4> [254.898927] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.898930] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.898932] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.898933] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.898939] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.899009] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.899011] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.899013] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.899015] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.899017] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.899018] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.899020] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.899021] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.899023] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.899025] CR2: 000000c00044f000 CR3: 0000000127fb6001 CR4: 0000000000f72ef0 <4> [254.899027] PKRU: 55555554 <4> [254.899028] Call Trace: <4> [254.899029] <TASK> <4> [254.899033] ? lock_sync+0x100/0x100 <4> [254.899038] ? lock_release+0xd0/0x2b0 <4> [254.899044] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.899049] process_one_work+0x239/0x760 <4> [254.899056] worker_thread+0x200/0x3f0 <4> [254.899059] ? __pfx_worker_thread+0x10/0x10 <4> [254.899062] kthread+0x10d/0x150 <4> [254.899065] ? __pfx_kthread+0x10/0x10 <4> [254.899069] ret_from_fork+0x3d4/0x480 <4> [254.899071] ? __pfx_kthread+0x10/0x10 <4> [254.899075] ret_from_fork_asm+0x1a/0x30 <4> [254.899082] </TASK> <4> [254.899083] irq event stamp: 359243 <4> [254.899085] hardirqs last enabled at (359249): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.899088] hardirqs last disabled at (359254): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.899090] softirqs last enabled at (358248): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.899093] softirqs last disabled at (358241): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.899096] ---[ end trace 0000000000000000 ]--- <5> [254.900071] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40658, lrc_seqno=40658, guc_id=0, flags=0x73 in no process [-1] <7> [254.900076] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.900181] ------------[ cut here ]------------ <4> [254.900183] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.900185] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.900265] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.900361] efi_pstore nfnetlink autofs4 <4> [254.900368] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.900371] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.900373] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.900376] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.900383] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.900468] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.900470] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.900473] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.900474] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.900476] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.900477] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.900479] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.900480] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.900482] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.900483] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.900485] PKRU: 55555554 <4> [254.900486] Call Trace: <4> [254.900488] <TASK> <4> [254.900492] ? lock_sync+0x100/0x100 <4> [254.900496] ? lock_release+0xd0/0x2b0 <4> [254.900502] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.900507] process_one_work+0x239/0x760 <4> [254.900513] worker_thread+0x200/0x3f0 <4> [254.900516] ? __pfx_worker_thread+0x10/0x10 <4> [254.900519] kthread+0x10d/0x150 <4> [254.900521] ? __pfx_kthread+0x10/0x10 <4> [254.900525] ret_from_fork+0x3d4/0x480 <4> [254.900534] ? __pfx_kthread+0x10/0x10 <4> [254.900538] ret_from_fork_asm+0x1a/0x30 <4> [254.900545] </TASK> <4> [254.900546] irq event stamp: 274921 <4> [254.900548] hardirqs last enabled at (274927): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.900550] hardirqs last disabled at (274932): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.900553] softirqs last enabled at (272342): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [254.900555] softirqs last disabled at (272340): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [254.900557] ---[ end trace 0000000000000000 ]--- <6> [254.900559] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <7> [254.900550] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <6> [254.900629] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.900634] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.900655] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.900900] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.901104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.901193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.901286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.901374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.901462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.901547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.901636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.901725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.901808] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.901920] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.902037] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.903063] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.912827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.913079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.914097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.914177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.914253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.914331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.914408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.914484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.914558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.914632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.914705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.914778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.914862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.914937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.915010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.915083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.915156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.915229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.915303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.915376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.915449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.915532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.915614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.915695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.915778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.915865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.915943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.916020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.916097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.916177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.916259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.916340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.916420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.916500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.916586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.916670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.916760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.916850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.916933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.917015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.917095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.917173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.917252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.917331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.917408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.917483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.917565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.917646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.917725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.917802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.917894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.917972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.918050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.918129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.918207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.918285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.918362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.918440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.918516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.918592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.918670] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.918781] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.918785] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40658, lrc_seqno=40658, guc_id=0, flags=0x73 in no process [-1] <7> [254.918787] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.918858] ------------[ cut here ]------------ <4> [254.918859] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.918860] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [254.918931] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.918994] efi_pstore nfnetlink autofs4 <4> [254.918998] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.919001] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.919002] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.919003] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.919008] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.919078] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.919080] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [254.919082] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.919084] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.919085] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.919087] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.919088] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.919090] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [254.919092] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.919093] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [254.919095] PKRU: 55555554 <4> [254.919096] Call Trace: <4> [254.919097] <TASK> <4> [254.919101] ? lock_sync+0x100/0x100 <4> [254.919106] ? lock_release+0xd0/0x2b0 <4> [254.919112] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.919117] process_one_work+0x239/0x760 <4> [254.919124] worker_thread+0x200/0x3f0 <4> [254.919127] ? __pfx_worker_thread+0x10/0x10 <4> [254.919129] kthread+0x10d/0x150 <4> [254.919132] ? __pfx_kthread+0x10/0x10 <4> [254.919136] ret_from_fork+0x3d4/0x480 <4> [254.919138] ? __pfx_kthread+0x10/0x10 <4> [254.919142] ret_from_fork_asm+0x1a/0x30 <4> [254.919149] </TASK> <4> [254.919150] irq event stamp: 277985 <4> [254.919152] hardirqs last enabled at (277991): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.919155] hardirqs last disabled at (277996): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.919157] softirqs last enabled at (276788): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.919160] softirqs last disabled at (276781): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.919162] ---[ end trace 0000000000000000 ]--- <6> [254.919164] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.919234] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.919239] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.920484] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.920601] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.920706] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.920911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.920997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.921091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.921180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.921266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.921349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.921433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.921519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.921600] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.921697] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.921814] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.922840] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.933564] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.933813] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.934849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.934929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.935005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.935082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.935159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.935234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.935310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.935385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.935459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.935536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.935613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.935688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.935762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.935840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.935915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.935988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.936062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.936135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.936210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.936296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.936378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.936456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.936536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.936613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.936691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.936768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.936849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.936929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.937010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.937088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.937167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.937247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.937330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.937413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.937498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.937579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.937658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.937736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.937813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.937902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.937982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.938059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.938136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.938216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.938299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.938377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.938454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.938531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.938608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.938682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.938757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.938838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.938913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.938992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.939073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.939154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.939231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.939307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.939387] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.939498] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.939502] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40658, lrc_seqno=40658, guc_id=0, flags=0x73 in no process [-1] <7> [254.939504] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.939563] ------------[ cut here ]------------ <4> [254.939564] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.939566] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [254.939638] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.939702] efi_pstore nfnetlink autofs4 <4> [254.939707] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.939710] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.939711] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.939713] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.939719] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.939788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.939790] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.939793] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.939794] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.939796] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.939797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.939799] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.939800] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [254.939802] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.939804] CR2: 000000c00044f000 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [254.939805] PKRU: 55555554 <4> [254.939806] Call Trace: <4> [254.939808] <TASK> <4> [254.939812] ? lock_sync+0x100/0x100 <4> [254.939817] ? lock_release+0xd0/0x2b0 <4> [254.939825] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.939831] process_one_work+0x239/0x760 <4> [254.939838] worker_thread+0x200/0x3f0 <4> [254.939841] ? __pfx_worker_thread+0x10/0x10 <4> [254.939843] kthread+0x10d/0x150 <4> [254.939846] ? __pfx_kthread+0x10/0x10 <4> [254.939850] ret_from_fork+0x3d4/0x480 <4> [254.939853] ? __pfx_kthread+0x10/0x10 <4> [254.939856] ret_from_fork_asm+0x1a/0x30 <4> [254.939864] </TASK> <4> [254.939865] irq event stamp: 363167 <4> [254.939866] hardirqs last enabled at (363173): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.939869] hardirqs last disabled at (363178): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.939872] softirqs last enabled at (362088): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.939874] softirqs last disabled at (362081): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.939877] ---[ end trace 0000000000000000 ]--- <5> [254.940676] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40713, lrc_seqno=40713, guc_id=0, flags=0x73 in no process [-1] <7> [254.940684] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.940803] ------------[ cut here ]------------ <4> [254.940805] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.940807] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [254.940915] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.940999] efi_pstore nfnetlink autofs4 <4> [254.941006] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.941012] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.941014] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.941016] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.941024] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.941106] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.941108] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.941110] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.941112] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.941113] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.941114] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.941116] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.941117] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.941119] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.941120] CR2: 00007e95d8473008 CR3: 0000000127fb6005 CR4: 0000000000f72ef0 <4> [254.941121] PKRU: 55555554 <4> [254.941122] Call Trace: <4> [254.941124] <TASK> <4> [254.941127] ? lock_sync+0x100/0x100 <4> [254.941132] ? lock_release+0xd0/0x2b0 <4> [254.941137] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.941143] process_one_work+0x239/0x760 <4> [254.941149] worker_thread+0x200/0x3f0 <4> [254.941152] ? __pfx_worker_thread+0x10/0x10 <4> [254.941155] kthread+0x10d/0x150 <4> [254.941158] ? __pfx_kthread+0x10/0x10 <4> [254.941161] ret_from_fork+0x3d4/0x480 <4> [254.941163] ? __pfx_kthread+0x10/0x10 <4> [254.941167] ret_from_fork_asm+0x1a/0x30 <4> [254.941174] </TASK> <4> [254.941175] irq event stamp: 490075 <4> [254.941176] hardirqs last enabled at (490081): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.941179] hardirqs last disabled at (490086): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.941182] softirqs last enabled at (487630): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [254.941184] softirqs last disabled at (487628): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [254.941186] ---[ end trace 0000000000000000 ]--- <7> [254.941161] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <6> [254.941188] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.941259] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.941266] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.941265] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [254.941452] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.941655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.941746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.941843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.941931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.942018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.942102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.942185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.942272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.942354] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.942449] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.942565] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.943576] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.953825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.954075] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.955105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.955184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.955259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.955334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.955410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.955487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.955565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.955641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.955716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.955790] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.955894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.955969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.956044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.956117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.956190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.956263] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.956335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.956408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.956481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.956562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.956643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.956725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.956808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.956902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.956980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.957057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.957137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.957218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.957299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.957378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.957456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.957534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.957618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.957702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.957785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.957876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.957959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.958038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.958115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.958191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.958269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.958346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.958424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.958500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.958579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.958655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.958733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.958808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.958898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.958973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.959048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.959126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.959206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.959286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.959363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.959440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.959515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.959589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.959668] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.959780] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.959784] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40713, lrc_seqno=40713, guc_id=0, flags=0x73 in no process [-1] <7> [254.959787] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.959860] ------------[ cut here ]------------ <4> [254.959861] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.959862] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [254.959935] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.959997] efi_pstore nfnetlink autofs4 <4> [254.960001] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.960004] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.960005] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.960006] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.960012] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.960082] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.960083] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.960086] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.960087] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.960088] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.960090] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.960091] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.960092] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.960094] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.960095] CR2: 00007e95d8473008 CR3: 0000000127fb6005 CR4: 0000000000f72ef0 <4> [254.960096] PKRU: 55555554 <4> [254.960098] Call Trace: <4> [254.960099] <TASK> <4> [254.960102] ? lock_sync+0x100/0x100 <4> [254.960107] ? lock_release+0xd0/0x2b0 <4> [254.960112] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.960118] process_one_work+0x239/0x760 <4> [254.960124] worker_thread+0x200/0x3f0 <4> [254.960127] ? __pfx_worker_thread+0x10/0x10 <4> [254.960129] kthread+0x10d/0x150 <4> [254.960132] ? __pfx_kthread+0x10/0x10 <4> [254.960135] ret_from_fork+0x3d4/0x480 <4> [254.960138] ? __pfx_kthread+0x10/0x10 <4> [254.960141] ret_from_fork_asm+0x1a/0x30 <4> [254.960148] </TASK> <4> [254.960149] irq event stamp: 493213 <4> [254.960150] hardirqs last enabled at (493219): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.960153] hardirqs last disabled at (493224): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.960155] softirqs last enabled at (492452): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.960158] softirqs last disabled at (492445): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.960160] ---[ end trace 0000000000000000 ]--- <6> [254.960162] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.960231] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.960238] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.960648] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.960856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.960944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.961035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.961124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.961212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.961298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.961384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.961470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.961551] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.961643] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.961761] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.962765] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.973492] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.973744] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.974780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.974870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.974945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.975022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.975098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.975173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.975247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.975321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.975394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.975467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.975543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.975618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.975693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.975768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.975848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.975923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.975997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.976070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.976143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.976224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.976304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.976383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.976466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.976545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.976623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.976699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.976776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [254.976865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [254.976945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [254.977022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [254.977101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [254.977180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [254.977267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [254.977352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [254.977436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [254.977518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [254.977597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [254.977673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [254.977749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [254.977828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [254.977906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [254.977985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [254.978060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [254.978139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [254.978222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [254.978300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [254.978377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [254.978454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [254.978531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [254.978607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [254.978683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [254.978761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [254.978840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [254.978916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [254.978991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [254.979067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [254.979142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [254.979217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [254.979295] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [254.979404] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [254.979408] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40713, lrc_seqno=40713, guc_id=0, flags=0x73 in no process [-1] <7> [254.979411] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.979470] ------------[ cut here ]------------ <4> [254.979471] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.979472] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:50/3229 <4> [254.979543] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.979605] efi_pstore nfnetlink autofs4 <4> [254.979610] CPU: 6 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.979612] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.979613] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.979615] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.979620] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.979689] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.979691] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [254.979693] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.979695] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.979697] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.979698] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.979699] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.979701] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [254.979703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.979704] CR2: 00007e95d8473008 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [254.979706] PKRU: 55555554 <4> [254.979707] Call Trace: <4> [254.979709] <TASK> <4> [254.979712] ? lock_sync+0x100/0x100 <4> [254.979717] ? lock_release+0xd0/0x2b0 <4> [254.979722] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.979728] process_one_work+0x239/0x760 <4> [254.979734] worker_thread+0x200/0x3f0 <4> [254.979737] ? __pfx_worker_thread+0x10/0x10 <4> [254.979740] kthread+0x10d/0x150 <4> [254.979743] ? __pfx_kthread+0x10/0x10 <4> [254.979747] ret_from_fork+0x3d4/0x480 <4> [254.979749] ? __pfx_kthread+0x10/0x10 <4> [254.979752] ret_from_fork_asm+0x1a/0x30 <4> [254.979760] </TASK> <4> [254.979761] irq event stamp: 496323 <4> [254.979762] hardirqs last enabled at (496329): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.979765] hardirqs last disabled at (496334): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.979767] softirqs last enabled at (495352): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.979770] softirqs last disabled at (495345): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.979772] ---[ end trace 0000000000000000 ]--- <7> [254.980830] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [254.980937] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [254.981475] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40768, lrc_seqno=40768, guc_id=0, flags=0x73 in no process [-1] <7> [254.981482] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [254.981590] ------------[ cut here ]------------ <4> [254.981592] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [254.981594] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#13: kworker/u64:25/3203 <4> [254.981711] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [254.981816] efi_pstore nfnetlink autofs4 <4> [254.981886] CPU: 13 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [254.981891] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [254.981893] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [254.981895] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [254.981904] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [254.982018] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [254.982020] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [254.982023] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [254.982025] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [254.982027] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [254.982029] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [254.982031] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [254.982033] FS: 0000000000000000(0000) GS:ffff8888db317000(0000) knlGS:0000000000000000 <4> [254.982035] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [254.982037] CR2: 0000773620d65000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [254.982040] PKRU: 55555554 <4> [254.982041] Call Trace: <4> [254.982043] <TASK> <4> [254.982049] ? lock_sync+0x100/0x100 <4> [254.982056] ? lock_release+0xd0/0x2b0 <4> [254.982065] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [254.982074] process_one_work+0x239/0x760 <4> [254.982085] worker_thread+0x200/0x3f0 <4> [254.982089] ? __pfx_worker_thread+0x10/0x10 <4> [254.982092] kthread+0x10d/0x150 <4> [254.982096] ? __pfx_kthread+0x10/0x10 <4> [254.982102] ret_from_fork+0x3d4/0x480 <4> [254.982105] ? __pfx_kthread+0x10/0x10 <4> [254.982110] ret_from_fork_asm+0x1a/0x30 <4> [254.982122] </TASK> <4> [254.982123] irq event stamp: 366981 <4> [254.982125] hardirqs last enabled at (366987): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [254.982129] hardirqs last disabled at (366992): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [254.982132] softirqs last enabled at (366766): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.982136] softirqs last disabled at (366545): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [254.982139] ---[ end trace 0000000000000000 ]--- <6> [254.982142] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [254.982256] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [254.982265] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [254.982490] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [254.982743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [254.982886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [254.983026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [254.983158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [254.983291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [254.983421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [254.983552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [254.983686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [254.983815] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [254.983962] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [254.984128] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [254.985505] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [254.995828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [254.996137] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [254.997320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [254.997441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [254.997558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [254.997678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [254.997797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [254.997925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [254.998046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [254.998165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [254.998284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [254.998403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [254.998522] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [254.998641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [254.998759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [254.998894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [254.998979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [254.999058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [254.999136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [254.999211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [254.999286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [254.999368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [254.999449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [254.999527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [254.999607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [254.999688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [254.999768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [254.999886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [254.999965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.000046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.000127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.000206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.000284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.000363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.000450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.000535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.000619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.000700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.000779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.000864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.000941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.001017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.001096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.001173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.001253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.001333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.001415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.001492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.001570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.001646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.001723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.001797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.001881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.001960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.002041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.002121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.002199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.002277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.002352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.002427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.002506] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.002616] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.002620] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40768, lrc_seqno=40768, guc_id=0, flags=0x73 in no process [-1] <7> [255.002622] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.002680] ------------[ cut here ]------------ <4> [255.002681] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.002683] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [255.002753] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.002834] efi_pstore nfnetlink autofs4 <4> [255.002839] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.002842] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.002843] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.002845] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.002850] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.002920] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.002921] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.002923] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.002925] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.002926] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.002927] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.002928] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.002930] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [255.002931] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.002933] CR2: 00007e95d8473008 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [255.002934] PKRU: 55555554 <4> [255.002935] Call Trace: <4> [255.002936] <TASK> <4> [255.002940] ? lock_sync+0x100/0x100 <4> [255.002945] ? lock_release+0xd0/0x2b0 <4> [255.002950] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.002956] process_one_work+0x239/0x760 <4> [255.002962] worker_thread+0x200/0x3f0 <4> [255.002965] ? __pfx_worker_thread+0x10/0x10 <4> [255.002967] kthread+0x10d/0x150 <4> [255.002970] ? __pfx_kthread+0x10/0x10 <4> [255.002974] ret_from_fork+0x3d4/0x480 <4> [255.002976] ? __pfx_kthread+0x10/0x10 <4> [255.002979] ret_from_fork_asm+0x1a/0x30 <4> [255.002986] </TASK> <4> [255.002988] irq event stamp: 370293 <4> [255.002989] hardirqs last enabled at (370299): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.002991] hardirqs last disabled at (370304): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.002994] softirqs last enabled at (369990): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.002996] softirqs last disabled at (369985): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.002998] ---[ end trace 0000000000000000 ]--- <6> [255.003932] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.004005] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.004014] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.004068] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.004168] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.004274] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.004476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.004564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.004656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.004743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.004832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.004917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.005001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.005087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.005168] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.005260] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.005377] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.006384] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.016825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.017073] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.018068] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.018146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.018221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.018297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.018373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.018450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.018526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.018602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.018676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.018750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.018827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.018901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.018977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.019053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.019128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.019202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.019276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.019350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.019424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.019504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.019585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.019666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.019747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.019829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.019905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.019982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.020057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.020136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.020215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.020292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.020371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.020454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.020540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.020625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.020710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.020791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.020880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.020957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.021034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.021111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.021189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.021266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.021342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.021417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.021496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.021572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.021648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.021727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.021807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.021897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.021973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.022052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.022128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.022205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.022283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.022363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.022439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.022515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.022594] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.022703] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.022707] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40768, lrc_seqno=40768, guc_id=0, flags=0x73 in no process [-1] <7> [255.022709] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.022768] ------------[ cut here ]------------ <4> [255.022769] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.022770] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [255.022856] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.022920] efi_pstore nfnetlink autofs4 <4> [255.022925] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.022928] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.022930] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.022931] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.022937] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.023007] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.023009] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.023011] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.023013] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.023014] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.023016] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.023017] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.023019] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [255.023020] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.023022] CR2: 00007e95d8473008 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [255.023024] PKRU: 55555554 <4> [255.023025] Call Trace: <4> [255.023026] <TASK> <4> [255.023030] ? lock_sync+0x100/0x100 <4> [255.023035] ? lock_release+0xd0/0x2b0 <4> [255.023041] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.023046] process_one_work+0x239/0x760 <4> [255.023053] worker_thread+0x200/0x3f0 <4> [255.023055] ? __pfx_worker_thread+0x10/0x10 <4> [255.023058] kthread+0x10d/0x150 <4> [255.023061] ? __pfx_kthread+0x10/0x10 <4> [255.023065] ret_from_fork+0x3d4/0x480 <4> [255.023067] ? __pfx_kthread+0x10/0x10 <4> [255.023071] ret_from_fork_asm+0x1a/0x30 <4> [255.023078] </TASK> <4> [255.023079] irq event stamp: 373453 <4> [255.023080] hardirqs last enabled at (373459): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.023083] hardirqs last disabled at (373464): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.023086] softirqs last enabled at (372710): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.023089] softirqs last disabled at (372653): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.023091] ---[ end trace 0000000000000000 ]--- <5> [255.023672] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40823, lrc_seqno=40823, guc_id=0, flags=0x73 in no process [-1] <7> [255.023675] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.023915] ------------[ cut here ]------------ <4> [255.023918] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.023920] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [255.024000] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.024082] efi_pstore nfnetlink autofs4 <4> [255.024088] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.024093] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.024095] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.024097] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.024105] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.024186] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.024188] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [255.024193] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.024195] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.024196] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.024198] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.024200] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.024202] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.024204] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.024206] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.024209] PKRU: 55555554 <4> [255.024211] Call Trace: <4> [255.024212] <TASK> <4> [255.024216] ? lock_sync+0x100/0x100 <4> [255.024224] ? lock_release+0xd0/0x2b0 <4> [255.024230] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.024237] process_one_work+0x239/0x760 <4> [255.024244] worker_thread+0x200/0x3f0 <4> [255.024247] ? __pfx_worker_thread+0x10/0x10 <4> [255.024250] kthread+0x10d/0x150 <4> [255.024255] ? __pfx_kthread+0x10/0x10 <4> [255.024259] ret_from_fork+0x3d4/0x480 <4> [255.024263] ? __pfx_kthread+0x10/0x10 <4> [255.024267] ret_from_fork_asm+0x1a/0x30 <4> [255.024275] </TASK> <4> [255.024277] irq event stamp: 501059 <4> [255.024278] hardirqs last enabled at (501065): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.024282] hardirqs last disabled at (501070): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.024285] softirqs last enabled at (499092): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [255.024288] softirqs last disabled at (499090): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [255.024291] ---[ end trace 0000000000000000 ]--- <6> [255.024295] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.024386] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.024401] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.024675] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.024781] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.024944] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.025183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.025282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.025383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.025477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.025569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.025661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.025754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.025853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.025943] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.026045] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.026167] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.027185] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.038164] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.038414] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.039405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.039482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.039559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.039637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.039713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.039788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.039874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.039948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.040022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.040098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.040172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.040247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.040321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.040394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.040470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.040547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.040622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.040695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.040769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.040857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.040940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.041021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.041102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.041182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.041263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.041340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.041417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.041497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.041577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.041655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.041741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.041823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.041907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.041988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.042075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.042159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.042239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.042317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.042394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.042469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.042549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.042626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.042703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.042779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.042869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.042946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.043022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.043096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.043174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.043249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.043327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.043408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.043487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.043565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.043641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.043718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.043793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.043878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.043956] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.044068] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.044072] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40823, lrc_seqno=40823, guc_id=0, flags=0x73 in no process [-1] <7> [255.044075] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.044133] ------------[ cut here ]------------ <4> [255.044134] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.044136] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [255.044206] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.044270] efi_pstore nfnetlink autofs4 <4> [255.044275] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.044278] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.044279] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.044281] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.044286] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.044356] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.044357] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [255.044360] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.044361] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.044363] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.044364] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.044365] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.044366] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.044368] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.044369] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.044371] PKRU: 55555554 <4> [255.044372] Call Trace: <4> [255.044373] <TASK> <4> [255.044377] ? lock_sync+0x100/0x100 <4> [255.044382] ? lock_release+0xd0/0x2b0 <4> [255.044388] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.044393] process_one_work+0x239/0x760 <4> [255.044399] worker_thread+0x200/0x3f0 <4> [255.044402] ? __pfx_worker_thread+0x10/0x10 <4> [255.044404] kthread+0x10d/0x150 <4> [255.044407] ? __pfx_kthread+0x10/0x10 <4> [255.044411] ret_from_fork+0x3d4/0x480 <4> [255.044413] ? __pfx_kthread+0x10/0x10 <4> [255.044416] ret_from_fork_asm+0x1a/0x30 <4> [255.044424] </TASK> <4> [255.044425] irq event stamp: 504183 <4> [255.044426] hardirqs last enabled at (504189): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.044429] hardirqs last disabled at (504194): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.044431] softirqs last enabled at (503334): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.044434] softirqs last disabled at (503327): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.044436] ---[ end trace 0000000000000000 ]--- <6> [255.044438] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.044508] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.044514] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.044920] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.045122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.045205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.045297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.045384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.045470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.045554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.045639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.045723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.045805] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.045912] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.046030] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.047050] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.057771] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.058028] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.059056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.059135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.059210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.059287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.059364] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.059440] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.059516] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.059589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.059663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.059736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.059810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.059896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.059970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.060045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.060122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.060198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.060272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.060346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.060421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.060501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.060583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.060662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.060745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.060827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.060905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.060984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.061061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.061139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.061218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.061294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.061376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.061458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.061543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.061628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.061712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.061792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.061880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.061957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.062036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.062116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.062199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.062279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.062357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.062434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.062514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.062590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.062667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.062742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.062822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.062897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.062970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.063047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.063121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.063197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.063272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.063348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.063423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.063498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.063580] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.063691] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.063695] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40823, lrc_seqno=40823, guc_id=0, flags=0x73 in no process [-1] <7> [255.063698] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.063756] ------------[ cut here ]------------ <4> [255.063757] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.063758] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [255.063832] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.063904] efi_pstore nfnetlink autofs4 <4> [255.063908] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.063911] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.063913] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.063914] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.063920] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.063990] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.063992] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [255.063994] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.063996] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.063997] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.063999] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.064000] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.064002] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.064003] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.064005] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.064007] PKRU: 55555554 <4> [255.064008] Call Trace: <4> [255.064009] <TASK> <4> [255.064013] ? lock_sync+0x100/0x100 <4> [255.064018] ? lock_release+0xd0/0x2b0 <4> [255.064024] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.064030] process_one_work+0x239/0x760 <4> [255.064036] worker_thread+0x200/0x3f0 <4> [255.064039] ? __pfx_worker_thread+0x10/0x10 <4> [255.064041] kthread+0x10d/0x150 <4> [255.064044] ? __pfx_kthread+0x10/0x10 <4> [255.064048] ret_from_fork+0x3d4/0x480 <4> [255.064050] ? __pfx_kthread+0x10/0x10 <4> [255.064054] ret_from_fork_asm+0x1a/0x30 <4> [255.064061] </TASK> <4> [255.064063] irq event stamp: 507281 <4> [255.064064] hardirqs last enabled at (507287): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.064067] hardirqs last disabled at (507292): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.064069] softirqs last enabled at (506286): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.064072] softirqs last disabled at (506281): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.064075] ---[ end trace 0000000000000000 ]--- <5> [255.065564] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40878, lrc_seqno=40878, guc_id=0, flags=0x73 in no process [-1] <7> [255.065567] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.065629] ------------[ cut here ]------------ <4> [255.065630] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.065631] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [255.065701] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.065763] efi_pstore nfnetlink autofs4 <4> [255.065767] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.065769] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.065770] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.065771] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.065776] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.065853] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.065855] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [255.065857] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.065858] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.065860] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.065861] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.065862] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.065864] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.065866] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.065867] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.065869] PKRU: 55555554 <4> [255.065870] Call Trace: <4> [255.065872] <TASK> <4> [255.065876] ? lock_sync+0x100/0x100 <4> [255.065880] ? lock_release+0xd0/0x2b0 <4> [255.065886] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.065891] process_one_work+0x239/0x760 <4> [255.065897] worker_thread+0x200/0x3f0 <4> [255.065900] ? __pfx_worker_thread+0x10/0x10 <4> [255.065903] kthread+0x10d/0x150 <4> [255.065906] ? __pfx_kthread+0x10/0x10 <4> [255.065910] ret_from_fork+0x3d4/0x480 <4> [255.065912] ? __pfx_kthread+0x10/0x10 <4> [255.065916] ret_from_fork_asm+0x1a/0x30 <4> [255.065923] </TASK> <4> [255.065924] irq event stamp: 509861 <4> [255.065925] hardirqs last enabled at (509867): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.065928] hardirqs last disabled at (509872): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.065931] softirqs last enabled at (509700): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.065933] softirqs last disabled at (509693): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.065936] ---[ end trace 0000000000000000 ]--- <7> [255.065949] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.066053] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [255.066151] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.066224] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.066230] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.066279] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.066487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.066579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.066674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.066762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.066855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.066941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.067029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.067118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.067201] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.067296] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.067412] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.068420] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.078825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.079077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.080103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.080184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.080261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.080340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.080418] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.080495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.080572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.080648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.080722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.080797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.080921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.080996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.081075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.081154] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.081231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.081307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.081381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.081456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.081531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.081615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.081697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.081776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.081869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.081946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.082022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.082098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.082174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.082252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.082332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.082414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.082495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.082576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.082661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.082744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.082833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.082914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.082994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.083072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.083150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.083228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.083311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.083392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.083469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.083545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.083625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.083702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.083781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.083868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.083946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.084021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.084098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.084177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.084252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.084327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.084402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.084479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.084554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.084629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.084707] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.084822] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.084829] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40878, lrc_seqno=40878, guc_id=0, flags=0x73 in no process [-1] <7> [255.084832] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.084890] ------------[ cut here ]------------ <4> [255.084892] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.084893] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.084964] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.085027] efi_pstore nfnetlink autofs4 <4> [255.085031] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.085034] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.085035] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.085037] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.085042] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.085112] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.085114] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.085116] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.085117] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.085119] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.085120] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.085121] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.085123] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.085124] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.085125] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.085127] PKRU: 55555554 <4> [255.085128] Call Trace: <4> [255.085129] <TASK> <4> [255.085133] ? lock_sync+0x100/0x100 <4> [255.085137] ? lock_release+0xd0/0x2b0 <4> [255.085143] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.085148] process_one_work+0x239/0x760 <4> [255.085155] worker_thread+0x200/0x3f0 <4> [255.085158] ? __pfx_worker_thread+0x10/0x10 <4> [255.085160] kthread+0x10d/0x150 <4> [255.085163] ? __pfx_kthread+0x10/0x10 <4> [255.085166] ret_from_fork+0x3d4/0x480 <4> [255.085168] ? __pfx_kthread+0x10/0x10 <4> [255.085172] ret_from_fork_asm+0x1a/0x30 <4> [255.085179] </TASK> <4> [255.085180] irq event stamp: 286685 <4> [255.085181] hardirqs last enabled at (286691): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.085184] hardirqs last disabled at (286696): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.085186] softirqs last enabled at (285582): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.085188] softirqs last disabled at (285573): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.085191] ---[ end trace 0000000000000000 ]--- <6> [255.085193] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.085262] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.085268] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.086506] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.086612] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.086719] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.086924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.087019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.087116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.087206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.087293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.087380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.087469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.087557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.087641] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.087737] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.087858] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.088867] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.098857] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.099104] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.100099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.100179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.100255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.100332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.100409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.100486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.100562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.100637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.100713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.100786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.100870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.100944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.101019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.101093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.101166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.101241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.101315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.101390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.101467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.101551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.101633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.101712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.101792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.101880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.101959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.102038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.102116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.102195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.102273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.102350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.102429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.102507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.102591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.102679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.102765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.102853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.102934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.103011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.103088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.103163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.103242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.103319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.103396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.103475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.103558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.103637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.103715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.103791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.103876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.103952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.104027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.104105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.104182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.104259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.104334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.104411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.104485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.104559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.104638] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.104748] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.104752] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40878, lrc_seqno=40878, guc_id=0, flags=0x73 in no process [-1] <7> [255.104755] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.104814] ------------[ cut here ]------------ <4> [255.104816] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.104820] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [255.104893] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.104956] efi_pstore nfnetlink autofs4 <4> [255.104962] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.104965] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.104966] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.104968] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.104973] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.105043] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.105045] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.105047] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.105049] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.105050] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.105051] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.105053] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.105054] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [255.105056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.105057] CR2: 00007e95d8473008 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [255.105059] PKRU: 55555554 <4> [255.105060] Call Trace: <4> [255.105061] <TASK> <4> [255.105065] ? lock_sync+0x100/0x100 <4> [255.105069] ? lock_release+0xd0/0x2b0 <4> [255.105075] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.105080] process_one_work+0x239/0x760 <4> [255.105087] worker_thread+0x200/0x3f0 <4> [255.105089] ? __pfx_worker_thread+0x10/0x10 <4> [255.105092] kthread+0x10d/0x150 <4> [255.105095] ? __pfx_kthread+0x10/0x10 <4> [255.105098] ret_from_fork+0x3d4/0x480 <4> [255.105100] ? __pfx_kthread+0x10/0x10 <4> [255.105104] ret_from_fork_asm+0x1a/0x30 <4> [255.105111] </TASK> <4> [255.105112] irq event stamp: 379821 <4> [255.105114] hardirqs last enabled at (379827): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.105116] hardirqs last disabled at (379832): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.105119] softirqs last enabled at (378818): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.105121] softirqs last disabled at (378811): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.105124] ---[ end trace 0000000000000000 ]--- <5> [255.106497] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40933, lrc_seqno=40933, guc_id=0, flags=0x73 in no process [-1] <7> [255.106500] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.106562] ------------[ cut here ]------------ <4> [255.106564] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.106565] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [255.106633] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.106695] efi_pstore nfnetlink autofs4 <4> [255.106699] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.106702] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.106703] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.106704] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.106709] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.106776] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.106778] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.106780] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.106781] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.106782] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.106784] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.106785] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.106786] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [255.106787] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.106789] CR2: 00007e95d8473008 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [255.106790] PKRU: 55555554 <4> [255.106791] Call Trace: <4> [255.106792] <TASK> <4> [255.106796] ? lock_sync+0x100/0x100 <4> [255.106800] ? lock_release+0xd0/0x2b0 <4> [255.106805] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.106811] process_one_work+0x239/0x760 <4> [255.106820] worker_thread+0x200/0x3f0 <4> [255.106823] ? __pfx_worker_thread+0x10/0x10 <4> [255.106825] kthread+0x10d/0x150 <4> [255.106828] ? __pfx_kthread+0x10/0x10 <4> [255.106832] ret_from_fork+0x3d4/0x480 <4> [255.106834] ? __pfx_kthread+0x10/0x10 <4> [255.106837] ret_from_fork_asm+0x1a/0x30 <4> [255.106845] </TASK> <4> [255.106846] irq event stamp: 382539 <4> [255.106847] hardirqs last enabled at (382545): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.106850] hardirqs last disabled at (382550): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.106852] softirqs last enabled at (378818): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.106855] softirqs last disabled at (378811): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.106857] ---[ end trace 0000000000000000 ]--- <7> [255.106870] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.106973] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [255.107070] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.107141] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.107147] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.107238] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.107445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.107537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.107634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.107724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.107814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.107919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.108008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.108097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.108178] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.108274] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.108389] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.109396] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.119825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.120074] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.121082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.121161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.121236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.121312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.121387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.121462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.121534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.121608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.121682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.121754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.121831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.121905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.121980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.122054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.122126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.122199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.122272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.122344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.122417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.122497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.122577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.122654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.122735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.122811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.122902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.122979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.123056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.123136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.123220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.123302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.123382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.123462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.123546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.123629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.123713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.123794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.123885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.123965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.124044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.124121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.124200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.124276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.124353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.124428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.124508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.124590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.124671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.124748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.124830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.124906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.124987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.125068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.125146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.125224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.125300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.125377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.125452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.125531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.125614] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.125725] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.125729] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40933, lrc_seqno=40933, guc_id=0, flags=0x73 in no process [-1] <7> [255.125731] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.125790] ------------[ cut here ]------------ <4> [255.125791] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.125792] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:50/3229 <4> [255.125875] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.125939] efi_pstore nfnetlink autofs4 <4> [255.125944] CPU: 10 UID: 0 PID: 3229 Comm: kworker/u64:50 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.125947] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.125948] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.125950] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.125955] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.126024] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.126026] RSP: 0018:ffffc900071c3ca0 EFLAGS: 00010246 <4> [255.126028] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.126030] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.126031] RBP: ffffc900071c3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.126032] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.126033] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.126035] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.126036] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.126038] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.126039] PKRU: 55555554 <4> [255.126040] Call Trace: <4> [255.126042] <TASK> <4> [255.126046] ? lock_sync+0x100/0x100 <4> [255.126050] ? lock_release+0xd0/0x2b0 <4> [255.126056] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.126061] process_one_work+0x239/0x760 <4> [255.126068] worker_thread+0x200/0x3f0 <4> [255.126071] ? __pfx_worker_thread+0x10/0x10 <4> [255.126073] kthread+0x10d/0x150 <4> [255.126076] ? __pfx_kthread+0x10/0x10 <4> [255.126080] ret_from_fork+0x3d4/0x480 <4> [255.126083] ? __pfx_kthread+0x10/0x10 <4> [255.126086] ret_from_fork_asm+0x1a/0x30 <4> [255.126094] </TASK> <4> [255.126095] irq event stamp: 515585 <4> [255.126096] hardirqs last enabled at (515591): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.126099] hardirqs last disabled at (515596): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.126102] softirqs last enabled at (514842): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.126104] softirqs last disabled at (514825): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.126107] ---[ end trace 0000000000000000 ]--- <6> [255.127035] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.127107] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.127113] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.127127] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.127229] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.127333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.127538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.127628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.127720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.127807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.127905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.127989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.128074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.128160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.128241] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.128335] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.128451] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.129456] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.139825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.140070] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.141016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.141095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.141172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.141250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.141328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.141404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.141479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.141554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.141628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.141702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.141776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.141860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.141935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.142011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.142088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.142165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.142239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.142313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.142389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.142469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.142549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.142628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.142711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.142790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.142878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.142955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.143032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.143111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.143191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.143269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.143347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.143425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.143508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.143591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.143674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.143757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.143843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.143921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.143999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.144074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.144153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.144229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.144305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.144380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.144459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.144533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.144609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.144685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.144763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.144847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.144926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.145007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.145085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.145162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.145237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.145314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.145388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.145468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.145551] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.145662] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.145667] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40933, lrc_seqno=40933, guc_id=0, flags=0x73 in no process [-1] <7> [255.145669] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.145728] ------------[ cut here ]------------ <4> [255.145730] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.145731] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:25/3203 <4> [255.145801] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.145869] efi_pstore nfnetlink autofs4 <4> [255.145873] CPU: 6 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.145876] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.145877] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.145878] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.145883] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.145952] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.145953] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.145956] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.145957] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.145958] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.145960] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.145961] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.145962] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [255.145964] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.145965] CR2: 00007e95d8473008 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [255.145966] PKRU: 55555554 <4> [255.145968] Call Trace: <4> [255.145969] <TASK> <4> [255.145972] ? lock_sync+0x100/0x100 <4> [255.145977] ? lock_release+0xd0/0x2b0 <4> [255.145983] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.145988] process_one_work+0x239/0x760 <4> [255.145994] worker_thread+0x200/0x3f0 <4> [255.145997] ? __pfx_worker_thread+0x10/0x10 <4> [255.145999] kthread+0x10d/0x150 <4> [255.146002] ? __pfx_kthread+0x10/0x10 <4> [255.146006] ret_from_fork+0x3d4/0x480 <4> [255.146008] ? __pfx_kthread+0x10/0x10 <4> [255.146011] ret_from_fork_asm+0x1a/0x30 <4> [255.146018] </TASK> <4> [255.146019] irq event stamp: 386173 <4> [255.146021] hardirqs last enabled at (386179): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.146023] hardirqs last disabled at (386184): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.146026] softirqs last enabled at (385178): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.146028] softirqs last disabled at (385171): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.146030] ---[ end trace 0000000000000000 ]--- <7> [255.147051] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.147175] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [255.147430] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40988, lrc_seqno=40988, guc_id=0, flags=0x73 in no process [-1] <7> [255.147433] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.147493] ------------[ cut here ]------------ <4> [255.147494] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.147496] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [255.147570] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.147635] efi_pstore nfnetlink autofs4 <4> [255.147641] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.147643] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.147645] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.147646] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.147651] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.147722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.147724] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.147727] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.147728] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.147729] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.147731] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.147732] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.147734] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.147736] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.147737] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.147739] PKRU: 55555554 <4> [255.147740] Call Trace: <4> [255.147741] <TASK> <4> [255.147745] ? lock_sync+0x100/0x100 <4> [255.147750] ? lock_release+0xd0/0x2b0 <4> [255.147755] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.147761] process_one_work+0x239/0x760 <4> [255.147767] worker_thread+0x200/0x3f0 <4> [255.147770] ? __pfx_worker_thread+0x10/0x10 <4> [255.147772] kthread+0x10d/0x150 <4> [255.147775] ? __pfx_kthread+0x10/0x10 <4> [255.147779] ret_from_fork+0x3d4/0x480 <4> [255.147781] ? __pfx_kthread+0x10/0x10 <4> [255.147784] ret_from_fork_asm+0x1a/0x30 <4> [255.147791] </TASK> <4> [255.147792] irq event stamp: 388795 <4> [255.147793] hardirqs last enabled at (388801): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.147796] hardirqs last disabled at (388806): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.147798] softirqs last enabled at (385178): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.147801] softirqs last disabled at (385171): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.147803] ---[ end trace 0000000000000000 ]--- <6> [255.147805] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.147917] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.147924] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.148160] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.148366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.148456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.148549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.148638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.148723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.148807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.148913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.149002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.149087] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.149183] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.149299] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.150310] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.160826] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.161080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.162095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.162174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.162249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.162325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.162401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.162478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.162555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.162631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.162706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.162779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.162867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.162942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.163018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.163093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.163167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.163241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.163315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.163388] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.163463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.163544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.163624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.163703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.163782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.163873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.163950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.164026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.164101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.164179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.164256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.164334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.164411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.164490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.164578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.164665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.164751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.164841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.164922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.164999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.165077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.165158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.165239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.165318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.165396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.165472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.165552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.165632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.165713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.165792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.165880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.165955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.166031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.166112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.166188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.166264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.166340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.166419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.166497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.166572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.166651] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.166761] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.166765] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40988, lrc_seqno=40988, guc_id=0, flags=0x73 in no process [-1] <7> [255.166768] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.166838] ------------[ cut here ]------------ <4> [255.166840] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.166841] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.166912] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.166975] efi_pstore nfnetlink autofs4 <4> [255.166979] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.166982] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.166983] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.166985] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.166990] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.167059] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.167061] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.167064] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.167065] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.167067] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.167068] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.167070] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.167071] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.167073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.167075] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.167077] PKRU: 55555554 <4> [255.167078] Call Trace: <4> [255.167079] <TASK> <4> [255.167083] ? lock_sync+0x100/0x100 <4> [255.167088] ? lock_release+0xd0/0x2b0 <4> [255.167093] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.167099] process_one_work+0x239/0x760 <4> [255.167105] worker_thread+0x200/0x3f0 <4> [255.167108] ? __pfx_worker_thread+0x10/0x10 <4> [255.167111] kthread+0x10d/0x150 <4> [255.167114] ? __pfx_kthread+0x10/0x10 <4> [255.167118] ret_from_fork+0x3d4/0x480 <4> [255.167120] ? __pfx_kthread+0x10/0x10 <4> [255.167124] ret_from_fork_asm+0x1a/0x30 <4> [255.167131] </TASK> <4> [255.167132] irq event stamp: 292609 <4> [255.167134] hardirqs last enabled at (292615): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.167136] hardirqs last disabled at (292620): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.167139] softirqs last enabled at (291848): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.167142] softirqs last disabled at (291839): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.167144] ---[ end trace 0000000000000000 ]--- <6> [255.167146] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.167216] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.167222] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.167627] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.167833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.167916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.168004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.168088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.168171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.168253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.168340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.168427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.168510] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.168604] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.168720] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.169728] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.179824] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.180071] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.181231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.181311] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.181386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.181461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.181537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.181613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.181688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.181762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.181844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.181921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.181995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.182069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.182144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.182218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.182291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.182365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.182439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.182514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.182587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.182667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.182748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.182835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.182915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.182991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.183067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.183143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.183222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.183304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.183385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.183464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.183543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.183622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.183710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.183796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.183890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.183972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.184052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.184129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.184205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.184281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.184360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.184438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.184515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.184591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.184670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.184746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.184828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.184905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.184982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.185058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.185132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.185212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.185293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.185372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.185449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.185526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.185601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.185676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.185754] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.185867] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.185871] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40988, lrc_seqno=40988, guc_id=0, flags=0x73 in no process [-1] <7> [255.185874] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.185932] ------------[ cut here ]------------ <4> [255.185933] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.185934] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.186005] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.186067] efi_pstore nfnetlink autofs4 <4> [255.186071] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.186074] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.186076] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.186077] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.186082] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.186153] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.186155] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.186157] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.186159] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.186161] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.186162] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.186163] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.186165] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.186167] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.186168] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.186170] PKRU: 55555554 <4> [255.186171] Call Trace: <4> [255.186173] <TASK> <4> [255.186177] ? lock_sync+0x100/0x100 <4> [255.186181] ? lock_release+0xd0/0x2b0 <4> [255.186187] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.186192] process_one_work+0x239/0x760 <4> [255.186198] worker_thread+0x200/0x3f0 <4> [255.186201] ? __pfx_worker_thread+0x10/0x10 <4> [255.186204] kthread+0x10d/0x150 <4> [255.186207] ? __pfx_kthread+0x10/0x10 <4> [255.186211] ret_from_fork+0x3d4/0x480 <4> [255.186213] ? __pfx_kthread+0x10/0x10 <4> [255.186217] ret_from_fork_asm+0x1a/0x30 <4> [255.186224] </TASK> <4> [255.186225] irq event stamp: 295683 <4> [255.186227] hardirqs last enabled at (295689): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.186229] hardirqs last disabled at (295694): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.186232] softirqs last enabled at (294848): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.186234] softirqs last disabled at (294843): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.186237] ---[ end trace 0000000000000000 ]--- <5> [255.187653] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41043, lrc_seqno=41043, guc_id=0, flags=0x73 in no process [-1] <7> [255.187655] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.187718] ------------[ cut here ]------------ <4> [255.187720] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.187721] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <7> [255.187717] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [255.187792] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.187872] efi_pstore <7> [255.187826] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [255.187874] nfnetlink autofs4 <4> [255.187878] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.187880] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.187881] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.187882] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.187887] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.187957] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.187959] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.187962] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.187963] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.187965] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.187966] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.187967] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.187969] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.187971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.187972] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.187974] PKRU: 55555554 <4> [255.187975] Call Trace: <4> [255.187976] <TASK> <4> [255.187980] ? lock_sync+0x100/0x100 <4> [255.187985] ? lock_release+0xd0/0x2b0 <4> [255.187990] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.187996] process_one_work+0x239/0x760 <4> [255.188002] worker_thread+0x200/0x3f0 <4> [255.188005] ? __pfx_worker_thread+0x10/0x10 <4> [255.188007] kthread+0x10d/0x150 <4> [255.188010] ? __pfx_kthread+0x10/0x10 <4> [255.188014] ret_from_fork+0x3d4/0x480 <4> [255.188016] ? __pfx_kthread+0x10/0x10 <4> [255.188020] ret_from_fork_asm+0x1a/0x30 <4> [255.188027] </TASK> <4> [255.188029] irq event stamp: 396501 <4> [255.188030] hardirqs last enabled at (396507): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.188033] hardirqs last disabled at (396512): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.188035] softirqs last enabled at (395962): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.188038] softirqs last disabled at (395957): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.188040] ---[ end trace 0000000000000000 ]--- <6> [255.188042] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.188112] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.188117] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.188295] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.188496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.188582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.188675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.188762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.188860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.188945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.189029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.189113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.189192] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.189285] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.189400] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.190407] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.200826] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.201078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.202063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.202142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.202218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.202294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.202368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.202443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.202517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.202591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.202667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.202744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.202824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.202901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.202975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.203049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.203123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.203196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.203269] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.203343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.203416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.203496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.203577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.203656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.203738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.203848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.203926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.204002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.204078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.204157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.204237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.204318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.204399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.204479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.204564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.204648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.204732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.204812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.204911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.204988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.205064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.205139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.205218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.205295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.205373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.205448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.205531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.205611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.205690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.205766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.205851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.205927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.206003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.206082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.206158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.206237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.206314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.206392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.206468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.206543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.206621] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.206731] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.206735] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41043, lrc_seqno=41043, guc_id=0, flags=0x73 in no process [-1] <7> [255.206738] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.206796] ------------[ cut here ]------------ <4> [255.206797] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.206798] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.206882] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.206946] efi_pstore nfnetlink autofs4 <4> [255.206950] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.206953] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.206954] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.206955] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.206960] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.207029] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.207030] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.207033] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.207034] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.207035] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.207037] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.207038] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.207039] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.207041] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.207042] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.207043] PKRU: 55555554 <4> [255.207045] Call Trace: <4> [255.207046] <TASK> <4> [255.207050] ? lock_sync+0x100/0x100 <4> [255.207054] ? lock_release+0xd0/0x2b0 <4> [255.207059] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.207065] process_one_work+0x239/0x760 <4> [255.207071] worker_thread+0x200/0x3f0 <4> [255.207074] ? __pfx_worker_thread+0x10/0x10 <4> [255.207076] kthread+0x10d/0x150 <4> [255.207079] ? __pfx_kthread+0x10/0x10 <4> [255.207083] ret_from_fork+0x3d4/0x480 <4> [255.207085] ? __pfx_kthread+0x10/0x10 <4> [255.207088] ret_from_fork_asm+0x1a/0x30 <4> [255.207095] </TASK> <4> [255.207096] irq event stamp: 300995 <4> [255.207098] hardirqs last enabled at (301001): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.207100] hardirqs last disabled at (301006): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.207103] softirqs last enabled at (300252): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.207105] softirqs last disabled at (300247): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.207107] ---[ end trace 0000000000000000 ]--- <6> [255.208032] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.208113] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.208120] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.208173] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.208271] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.208373] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.208575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.208663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.208753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.208848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.208933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.209016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.209100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.209186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.209267] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.209360] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.209477] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.210481] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.220830] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.221078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.221997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.222079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.222155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.222233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.222310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.222385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.222460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.222533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.222607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.222681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.222755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.222835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.222909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.222982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.223060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.223136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.223211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.223286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.223360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.223441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.223522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.223601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.223682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.223760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.223843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.223920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.223996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.224075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.224153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.224230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.224311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.224393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.224478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.224562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.224645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.224727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.224807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.224897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.224975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.225050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.225128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.225205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.225282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.225362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.225446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.225524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.225602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.225679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.225756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.225841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.225920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.226002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.226079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.226157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.226232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.226308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.226383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.226458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.226537] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.226647] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.226651] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41043, lrc_seqno=41043, guc_id=0, flags=0x73 in no process [-1] <7> [255.226654] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.226712] ------------[ cut here ]------------ <4> [255.226713] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.226714] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.226785] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.226864] efi_pstore nfnetlink autofs4 <4> [255.226868] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.226871] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.226873] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.226874] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.226880] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.226951] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.226953] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.226955] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.226956] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.226958] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.226959] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.226960] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.226962] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.226963] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.226965] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.226966] PKRU: 55555554 <4> [255.226967] Call Trace: <4> [255.226968] <TASK> <4> [255.226972] ? lock_sync+0x100/0x100 <4> [255.226977] ? lock_release+0xd0/0x2b0 <4> [255.226983] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.226988] process_one_work+0x239/0x760 <4> [255.226994] worker_thread+0x200/0x3f0 <4> [255.226997] ? __pfx_worker_thread+0x10/0x10 <4> [255.227000] kthread+0x10d/0x150 <4> [255.227003] ? __pfx_kthread+0x10/0x10 <4> [255.227006] ret_from_fork+0x3d4/0x480 <4> [255.227008] ? __pfx_kthread+0x10/0x10 <4> [255.227011] ret_from_fork_asm+0x1a/0x30 <4> [255.227019] </TASK> <4> [255.227020] irq event stamp: 304097 <4> [255.227021] hardirqs last enabled at (304103): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.227023] hardirqs last disabled at (304108): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.227026] softirqs last enabled at (303574): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.227028] softirqs last disabled at (303569): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.227030] ---[ end trace 0000000000000000 ]--- <7> [255.228567] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.228675] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [255.228773] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41098, lrc_seqno=41098, guc_id=0, flags=0x73 in no process [-1] <7> [255.228775] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.228846] ------------[ cut here ]------------ <4> [255.228847] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.228849] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.228920] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.228994] efi_pstore nfnetlink autofs4 <4> [255.228998] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.229001] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.229003] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.229004] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.229009] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.229078] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.229080] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.229083] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.229084] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.229086] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.229087] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.229089] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.229090] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.229092] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.229093] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.229095] PKRU: 55555554 <4> [255.229096] Call Trace: <4> [255.229097] <TASK> <4> [255.229101] ? lock_sync+0x100/0x100 <4> [255.229105] ? lock_release+0xd0/0x2b0 <4> [255.229110] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.229116] process_one_work+0x239/0x760 <4> [255.229122] worker_thread+0x200/0x3f0 <4> [255.229125] ? __pfx_worker_thread+0x10/0x10 <4> [255.229127] kthread+0x10d/0x150 <4> [255.229130] ? __pfx_kthread+0x10/0x10 <4> [255.229133] ret_from_fork+0x3d4/0x480 <4> [255.229135] ? __pfx_kthread+0x10/0x10 <4> [255.229139] ret_from_fork_asm+0x1a/0x30 <4> [255.229146] </TASK> <4> [255.229147] irq event stamp: 306857 <4> [255.229148] hardirqs last enabled at (306863): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.229151] hardirqs last disabled at (306868): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.229153] softirqs last enabled at (305238): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70 <4> [255.229155] softirqs last disabled at (305236): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120 <4> [255.229158] ---[ end trace 0000000000000000 ]--- <6> [255.229160] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.229229] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.229235] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.229317] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.229525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.229620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.229716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.229807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.229908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.229997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.230087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.230178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.230263] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.230357] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.230474] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.231484] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.241863] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.242113] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.243135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.243213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.243289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.243368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.243446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.243521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.243595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.243669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.243743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.243821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.243895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.243969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.244041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.244114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.244187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.244260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.244333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.244406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.244482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.244566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.244647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.244724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.244803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.244891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.244968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.245043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.245119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.245197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.245276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.245353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.245434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.245515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.245600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.245684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.245768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.245857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.245941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.246021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.246100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.246175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.246254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.246331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.246408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.246483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.246565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.246646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.246725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.246802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.246892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.246968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.247043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.247120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.247197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.247275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.247355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.247437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.247512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.247588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.247668] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.247779] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.247783] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41098, lrc_seqno=41098, guc_id=0, flags=0x73 in no process [-1] <7> [255.247785] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.248710] ------------[ cut here ]------------ <4> [255.248711] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.248713] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [255.248785] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.248853] efi_pstore nfnetlink autofs4 <4> [255.248858] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.248861] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.248862] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.248864] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.248869] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.248939] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.248941] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.248943] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.248945] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.248946] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.248947] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.248948] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.248949] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.248951] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.248952] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.248954] PKRU: 55555554 <4> [255.248955] Call Trace: <4> [255.248956] <TASK> <4> [255.248960] ? lock_sync+0x100/0x100 <4> [255.248965] ? lock_release+0xd0/0x2b0 <4> [255.248970] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.248976] process_one_work+0x239/0x760 <4> [255.248982] worker_thread+0x200/0x3f0 <4> [255.248985] ? __pfx_worker_thread+0x10/0x10 <4> [255.248987] kthread+0x10d/0x150 <4> [255.248990] ? __pfx_kthread+0x10/0x10 <4> [255.248994] ret_from_fork+0x3d4/0x480 <4> [255.248996] ? __pfx_kthread+0x10/0x10 <4> [255.248999] ret_from_fork_asm+0x1a/0x30 <4> [255.249006] </TASK> <4> [255.249007] irq event stamp: 402177 <4> [255.249009] hardirqs last enabled at (402183): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.249011] hardirqs last disabled at (402188): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.249013] softirqs last enabled at (401688): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.249016] softirqs last disabled at (401681): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.249019] ---[ end trace 0000000000000000 ]--- <6> [255.249020] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.249093] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.249099] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.249113] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.249214] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.249510] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.249716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.249807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.249919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.250006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.250092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.250176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.250262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.250350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.250434] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.250529] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.250643] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.251653] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.261832] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.262081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.263055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.263135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.263210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.263287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.263362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.263436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.263510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.263583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.263656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.263734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.263809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.263898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.263974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.264048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.264121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.264195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.264267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.264340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.264413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.264493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.264575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.264653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.264732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.264808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.264902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.264979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.265054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.265136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.265218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.265297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.265375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.265453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.265536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.265618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.265700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.265780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.265866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.265943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.266019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.266092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.266170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.266246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.266323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.266398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.266477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.266553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.266630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.266709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.266789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.266879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.266956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.267036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.267116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.267196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.267272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.267348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.267423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.267498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.267577] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.267688] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.267692] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41098, lrc_seqno=41098, guc_id=0, flags=0x73 in no process [-1] <7> [255.267694] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.267753] ------------[ cut here ]------------ <4> [255.267754] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.267755] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.267835] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.267898] efi_pstore nfnetlink autofs4 <4> [255.267902] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.267905] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.267906] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.267907] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.267913] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.267983] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.267985] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.267987] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.267989] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.267990] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.267992] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.267993] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.267995] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.267996] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.267998] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.267999] PKRU: 55555554 <4> [255.268001] Call Trace: <4> [255.268002] <TASK> <4> [255.268006] ? lock_sync+0x100/0x100 <4> [255.268011] ? lock_release+0xd0/0x2b0 <4> [255.268016] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.268022] process_one_work+0x239/0x760 <4> [255.268028] worker_thread+0x200/0x3f0 <4> [255.268031] ? __pfx_worker_thread+0x10/0x10 <4> [255.268034] kthread+0x10d/0x150 <4> [255.268037] ? __pfx_kthread+0x10/0x10 <4> [255.268040] ret_from_fork+0x3d4/0x480 <4> [255.268043] ? __pfx_kthread+0x10/0x10 <4> [255.268046] ret_from_fork_asm+0x1a/0x30 <4> [255.268053] </TASK> <4> [255.268055] irq event stamp: 310519 <4> [255.268056] hardirqs last enabled at (310525): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.268059] hardirqs last disabled at (310530): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.268062] softirqs last enabled at (309776): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.268064] softirqs last disabled at (309745): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.268067] ---[ end trace 0000000000000000 ]--- <5> [255.269608] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41153, lrc_seqno=41153, guc_id=0, flags=0x73 in no process [-1] <7> [255.269611] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.269673] ------------[ cut here ]------------ <4> [255.269674] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.269676] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.269746] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.269808] efi_pstore nfnetlink autofs4 <4> [255.269812] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.269814] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.269815] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.269823] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.269830] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.269899] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.269901] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.269903] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.269905] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.269906] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.269908] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.269909] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.269911] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.269912] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.269914] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.269916] PKRU: 55555554 <4> [255.269917] Call Trace: <4> [255.269918] <TASK> <4> [255.269922] ? lock_sync+0x100/0x100 <4> [255.269927] ? lock_release+0xd0/0x2b0 <4> [255.269932] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.269938] process_one_work+0x239/0x760 <4> [255.269944] worker_thread+0x200/0x3f0 <4> [255.269947] ? __pfx_worker_thread+0x10/0x10 <4> [255.269949] kthread+0x10d/0x150 <4> [255.269952] ? __pfx_kthread+0x10/0x10 <4> [255.269956] ret_from_fork+0x3d4/0x480 <4> [255.269958] ? __pfx_kthread+0x10/0x10 <4> [255.269962] ret_from_fork_asm+0x1a/0x30 <4> [255.269969] </TASK> <4> [255.269971] irq event stamp: 313319 <4> [255.269972] hardirqs last enabled at (313325): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.269975] hardirqs last disabled at (313330): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.269977] softirqs last enabled at (309776): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.269980] softirqs last disabled at (309745): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.269982] ---[ end trace 0000000000000000 ]--- <6> [255.269984] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.270054] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.270060] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.270292] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.270395] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.270501] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.270703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.270791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.270896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.270985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.271070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.271153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.271237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.271322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.271404] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.271498] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.271617] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.272629] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.282827] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.283080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.284092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.284173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.284248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.284324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.284399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.284473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.284547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.284621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.284698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.284774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.284857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.284932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.285007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.285083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.285158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.285232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.285306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.285378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.285450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.285530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.285610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.285689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.285768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.285853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.285933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.286012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.286089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.286168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.286248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.286326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.286405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.286483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.286566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.286648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.286731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.286812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.286911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.286989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.287070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.287148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.287229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.287308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.287384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.287460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.287540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.287616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.287692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.287767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.287858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.287937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.288014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.288093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.288170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.288247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.288322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.288399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.288474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.288549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.288627] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.288737] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.288741] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41153, lrc_seqno=41153, guc_id=0, flags=0x73 in no process [-1] <7> [255.288743] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.288801] ------------[ cut here ]------------ <4> [255.288802] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.288804] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.288902] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.288979] efi_pstore nfnetlink autofs4 <4> [255.288985] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.288988] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.288989] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.288991] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.288996] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.289067] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.289069] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.289071] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.289073] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.289074] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.289076] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.289077] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.289079] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.289080] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.289082] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.289083] PKRU: 55555554 <4> [255.289084] Call Trace: <4> [255.289085] <TASK> <4> [255.289089] ? lock_sync+0x100/0x100 <4> [255.289093] ? lock_release+0xd0/0x2b0 <4> [255.289099] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.289104] process_one_work+0x239/0x760 <4> [255.289110] worker_thread+0x200/0x3f0 <4> [255.289113] ? __pfx_worker_thread+0x10/0x10 <4> [255.289115] kthread+0x10d/0x150 <4> [255.289118] ? __pfx_kthread+0x10/0x10 <4> [255.289122] ret_from_fork+0x3d4/0x480 <4> [255.289124] ? __pfx_kthread+0x10/0x10 <4> [255.289127] ret_from_fork_asm+0x1a/0x30 <4> [255.289134] </TASK> <4> [255.289136] irq event stamp: 316413 <4> [255.289137] hardirqs last enabled at (316419): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.289139] hardirqs last disabled at (316424): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.289142] softirqs last enabled at (315670): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.289144] softirqs last disabled at (315663): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.289146] ---[ end trace 0000000000000000 ]--- <6> [255.290295] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.290376] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.290384] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.290461] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.290564] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.290670] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.290877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.290965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.291055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.291145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.291231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.291314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.291399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.291484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.291565] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.291660] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.291775] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.292784] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.302825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.303071] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.304081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.304158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.304232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.304310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.304389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.304464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.304539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.304613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.304688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.304762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.304847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.304920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.304994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.305067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.305140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.305212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.305285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.305358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.305431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.305511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.305591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.305669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.305749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.305833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.305915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.305995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.306071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.306150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.306228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.306306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.306384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.306461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.306544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.306627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.306710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.306789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.306875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.306954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.307035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.307113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.307194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.307272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.307349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.307425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.307505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.307580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.307656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.307731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.307807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.307899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.307977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.308057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.308133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.308208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.308284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.308360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.308434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.308509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.308586] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.308696] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.308700] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41153, lrc_seqno=41153, guc_id=0, flags=0x73 in no process [-1] <7> [255.308702] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.308760] ------------[ cut here ]------------ <4> [255.308761] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.308763] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/3209 <4> [255.308839] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.308903] efi_pstore nfnetlink autofs4 <4> [255.308907] CPU: 0 UID: 0 PID: 3209 Comm: kworker/u64:31 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.308910] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.308911] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.308912] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.308917] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.308987] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.308989] RSP: 0018:ffffc90005703ca0 EFLAGS: 00010246 <4> [255.308992] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.308993] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.308995] RBP: ffffc90005703db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.308996] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.308997] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.308999] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [255.309001] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.309002] CR2: 0000773620d60000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [255.309004] PKRU: 55555554 <4> [255.309005] Call Trace: <4> [255.309006] <TASK> <4> [255.309011] ? lock_sync+0x100/0x100 <4> [255.309015] ? lock_release+0xd0/0x2b0 <4> [255.309021] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.309027] process_one_work+0x239/0x760 <4> [255.309033] worker_thread+0x200/0x3f0 <4> [255.309036] ? __pfx_worker_thread+0x10/0x10 <4> [255.309038] kthread+0x10d/0x150 <4> [255.309041] ? __pfx_kthread+0x10/0x10 <4> [255.309045] ret_from_fork+0x3d4/0x480 <4> [255.309047] ? __pfx_kthread+0x10/0x10 <4> [255.309051] ret_from_fork_asm+0x1a/0x30 <4> [255.309058] </TASK> <4> [255.309059] irq event stamp: 319523 <4> [255.309061] hardirqs last enabled at (319529): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.309064] hardirqs last disabled at (319534): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.309066] softirqs last enabled at (318780): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.309069] softirqs last disabled at (318769): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.309071] ---[ end trace 0000000000000000 ]--- <5> [255.309724] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41208, lrc_seqno=41208, guc_id=0, flags=0x73 in no process [-1] <7> [255.309729] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.309883] ------------[ cut here ]------------ <4> [255.309885] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.309887] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [255.309976] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.310051] efi_pstore nfnetlink autofs4 <4> [255.310056] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.310061] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.310063] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [255.310066] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [255.310072] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [255.310152] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [255.310157] RSP: 0018:ffffc900056d3ca0 EFLAGS: 00010246 <4> [255.310161] RAX: ffffffffa1202fc9 RBX: 0000000000000000 RCX: 0000000000000000 <4> [255.310164] RDX: ffff888104ab7410 RSI: ffffffffa1202fc9 RDI: ffffffffa1003ef0 <4> [255.310166] RBP: ffffc900056d3db0 R08: 0000000000000000 R09: 0000000000000000 <4> [255.310167] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [255.310168] R13: ffff888104ab7410 R14: ffff88812db75018 R15: 00000000ffffffc2 <4> [255.310170] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [255.310172] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [255.310175] CR2: 000000c00044f000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [255.310176] PKRU: 55555554 <4> [255.310178] Call Trace: <4> [255.310180] <TASK> <4> [255.310185] ? lock_sync+0x100/0x100 <4> [255.310191] ? lock_release+0xd0/0x2b0 <4> [255.310198] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [255.310204] process_one_work+0x239/0x760 <4> [255.310211] worker_thread+0x200/0x3f0 <4> [255.310214] ? __pfx_worker_thread+0x10/0x10 <4> [255.310217] kthread+0x10d/0x150 <4> [255.310222] ? __pfx_kthread+0x10/0x10 <4> [255.310226] ret_from_fork+0x3d4/0x480 <4> [255.310229] ? __pfx_kthread+0x10/0x10 <4> [255.310233] ret_from_fork_asm+0x1a/0x30 <4> [255.310242] </TASK> <4> [255.310244] irq event stamp: 405117 <4> [255.310245] hardirqs last enabled at (405123): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0 <4> [255.310249] hardirqs last disabled at (405128): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0 <4> [255.310253] softirqs last enabled at (404356): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.310256] softirqs last disabled at (404175): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [255.310258] ---[ end trace 0000000000000000 ]--- <6> [255.310262] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [255.310347] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [255.310360] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [255.310512] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [255.310617] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [255.310722] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [255.310938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [255.311027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [255.311118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [255.311206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [255.311293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [255.311378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [255.311466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [255.311556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [255.311640] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [255.311735] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [255.311853] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [255.312867] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [255.322825] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [255.323077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [255.324065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [255.324145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [255.324221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [255.324297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [255.324372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [255.324446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [255.324521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [255.324597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [255.324673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [255.324750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [255.324838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [255.324913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [255.324988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [255.325062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [255.325136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [255.325209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [255.325283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [255.325356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [255.325429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [255.325512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [255.325594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [255.325674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [255.325757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [255.325842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [255.325921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [255.326000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [255.326077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [255.326156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [255.326236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [255.326314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [255.326394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [255.326472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [255.326558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [255.326645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [255.326732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [255.326824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [255.326911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [255.326992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [255.327071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [255.327147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [255.327226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [255.327304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [255.327384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [255.327464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [255.327547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [255.327625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [255.327702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [255.327778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [255.327863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [255.327938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [255.328012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [255.328090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [255.328167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [255.328243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [255.328317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [255.328394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [255.328473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [255.328551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [255.328633] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [255.328742] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [255.328746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41208, lrc_seqno=41208, guc_id=0, flags=0x73 in no process [-1] <7> [255.328749] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [255.328808] ------------[ cut here ]------------ <4> [255.328809] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [255.328811] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:25/3203 <4> [255.328896] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse <4> [255.328960] efi_pstore nfnetlink autofs4 <4> [255.328965] CPU: 10 UID: 0 PID: 3203 Comm: kworker/u64:25 Tainted: G S U W L 7.0.0-rc7-lgci-xe-xe-pw-164469v1-debug+ #1 PREEMPT(lazy) <4> [255.328967] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [255.328969] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024

<4> [242.682451] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [242.683381] Code: b6 79 26 48 89 55 c8 88 45 d0 e8 84 e9 5c e1 48 89 c6 48 8d 3d ea ba 37 00 0f b6 4d d0 41 54 45 89 e9 45 0f b6 c7 48 8b 55 c8 <67> 48 0f b9 3a 58 eb 91 41 0f b6 bd 99 0f 00 00 e8 14 e0 03 00 41

<4> [243.327040] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [243.327857] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [248.827572] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [248.828990] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [253.946095] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [253.946937] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [253.950773] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [253.950922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [253.973162] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [253.973311] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [253.993795] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [253.993949] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [253.994904] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [253.995084] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.015774] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.015932] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.034887] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.035042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.035996] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.036216] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.054906] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.055054] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.074865] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.075018] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.075973] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.076164] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.094895] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.095044] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.114627] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.114775] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.115817] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.116032] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.134897] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.135046] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.154797] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.154954] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.155745] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.155907] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.175975] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.176125] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.196787] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.196944] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.197851] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.198083] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.220881] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.221106] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.241897] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.242047] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.243020] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.243261] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.262847] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.263000] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.282478] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.282624] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.284720] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.284877] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.303498] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.303633] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.322690] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.322834] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.323637] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.323788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.343685] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.343854] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.363863] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.364011] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.365067] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.365357] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.385010] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof

<4> [254.385045] usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4

<4> [254.385161] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.403888] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.404036] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.404838] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.404983] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.426411] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.426563] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.446818] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.446981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.447953] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.448182] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.467776] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.467929] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.486926] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.487074] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.488092] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.488282] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.507806] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.507958] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.526898] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.527045] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.528053] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.528267] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.546813] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.546965] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.565836] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.565984] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.568056] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.568283] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.590042] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.590196] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.610816] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.610970] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.611872] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.612065] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.630828] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.630977] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.651775] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.651929] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.652733] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp

<4> [254.652756] hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4

<4> [254.652922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.671883] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.672036] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.691832] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.691988] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.693206] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.693398] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.711960] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.712114] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.731804] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.731962] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.733160] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.733316] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.756482] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.756647] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.776444] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.776597] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.777870] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.778069] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.796887] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.797042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.816434] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.816580] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.818364] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.818512] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.836841] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.836990] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.857522] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.857676] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.859663] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.859824] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.878883] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.879031] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.898855] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.899009] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.900265] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.900468] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.918931] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.919078] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.939638] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.939788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.940915] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.941106] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.959935] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.960082] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.979543] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.979689] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [254.981711] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [254.982018] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.002753] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.002920] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.022856] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.023007] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.024000] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.024186] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.044206] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.044356] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.063832] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.063990] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.065701] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.065853] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.084964] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.085112] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.104893] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.105043] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.106633] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.106776] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.125875] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.126024] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.145801] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.145952] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.147570] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.147722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.166912] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.167059] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.186005] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.186153] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.187792] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.187957] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.206882] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.207029] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.226785] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.226951] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.228920] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.229078] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.248785] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.248939] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.267835] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.267983] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.269746] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.269899] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.288902] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.289067] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.308839] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.308987] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.309976] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

<4> [255.310152] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d3 5e e1 48 89 c6 48 8d 3d ac 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [255.328896] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic cmdlinepart coretemp asus_nb_wmi spi_nor asus_wmi mei_pxp mei_hdcp sparse_keymap platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel snd_hda_intel rapl snd_intel_dspcfg snd_hda_codec r8169 intel_cstate snd_hda_core snd_hwdep video snd_pcm realtek snd_timer i2c_i801 mei_me idma64 i2c_mux snd spi_intel_pci soundcore i2c_smbus mei spi_intel intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec wmi acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse

Results for igt@xe_pat@pt-caching