Result: 415 Warning(s)
i915_display_info7 igt_runner7 results7.json results7-xe-load.json guc_logs7.tar i915_display_info_post_exec7 boot7 dmesg7
| Detail | Value |
|---|---|
| Duration | unknown |
| Hostname |
shard-bmg-9 |
| Igt-Version |
IGT-Version: 2.4-g63a08403d (x86_64) (Linux: 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ x86_64) |
| Out |
Using IGT_SRANDOM=1776303494 for randomisation Opened device: /dev/dri/card0 Test requirement not met in function __igt_unique____real_main2210, file ../tests/intel/xe_pat.c:2298: Test requirement: intel_graphics_ver(dev_id) == IP_VER(35, 10) Last errno: 2, No such file or directory Starting subtest: pt-caching va_bits: 48, num_objs: 16384, spread over pt levels: [0]: 16384 [1]: 512 [2]: 16 Subtest pt-caching: SUCCESS (12.979s) This test caused an abort condition: Kernel badly tainted (0x244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened. |
| Err |
Starting subtest: pt-caching Subtest pt-caching: SUCCESS (12.979s) |
| Dmesg |
<6> [580.104428] Console: switching to colour dummy device 80x25
<6> [580.104715] [IGT] xe_pat: executing
<6> [580.126359] [IGT] xe_pat: starting subtest pt-caching
<6> [581.681472] xe 0000:03:00.0: [drm] Tile0: GT0:
ASID: 0
Faulted Address: 0x00000002fb436000
FaultType: 0
AccessType: 0
FaultLevel: 2
EngineClass: 3 bcs
EngineInstance: 8
<6> [581.681488] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL
<6> [581.681534] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0
<7> [581.682137] xe 0000:03:00.0: [drm:xe_hw_engine_snapshot_capture [xe]] Tile0: GT0: Proceeding with manual engine snapshot
<6> [581.682606] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249
<5> [581.682615] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51137, lrc_seqno=51137, guc_id=0, flags=0x73 in no process [-1]
<6> [581.860877] xe 0000:03:00.0: [drm] Xe device coredump has been created
<6> [581.860880] xe 0000:03:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data
<4> [581.860882] ------------[ cut here ]------------
<4> [581.860884] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.860885] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:31/12306
<4> [581.861000] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.861072] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.861078] CPU: 11 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.861081] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER
<4> [581.861082] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.861084] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.861090] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.861155] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.861156] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [581.861159] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.861160] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.861161] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.861162] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.861163] R13: ffff888104082010 R14: ffff888132ea0000 R15: 00000000fffffffb
<4> [581.861165] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [581.861166] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.861167] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [581.861169] PKRU: 55555554
<4> [581.861170] Call Trace:
<4> [581.861171] <TASK>
<4> [581.861175] ? lock_sync+0x100/0x100
<4> [581.861181] ? __pfx_autoremove_wake_function+0x10/0x10
<4> [581.861186] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.861191] process_one_work+0x239/0x760
<4> [581.861197] worker_thread+0x200/0x3f0
<4> [581.861199] ? __pfx_worker_thread+0x10/0x10
<4> [581.861202] kthread+0x10d/0x150
<4> [581.861205] ? __pfx_kthread+0x10/0x10
<4> [581.861208] ret_from_fork+0x3d4/0x480
<4> [581.861210] ? __pfx_kthread+0x10/0x10
<4> [581.861213] ret_from_fork_asm+0x1a/0x30
<4> [581.861220] </TASK>
<4> [581.861222] irq event stamp: 402879
<4> [581.861223] hardirqs last enabled at (402885): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.861226] hardirqs last disabled at (402890): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.861228] softirqs last enabled at (401956): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.861231] softirqs last disabled at (401951): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.861233] ---[ end trace 0000000000000000 ]---
<6> [581.861235] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.861314] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.861365] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.861716] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.862393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.862486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.862573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.862658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.862739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.862818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.862897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.862977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.863052] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.863139] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.863246] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.864398] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [581.874321] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [581.874590] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [581.876122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [581.876208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [581.876296] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [581.876378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [581.876454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [581.876533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [581.876609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [581.876683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [581.876756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [581.876829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [581.876902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [581.876975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [581.877047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [581.877119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [581.877193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [581.877271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [581.877349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [581.877421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [581.877495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [581.877576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [581.877656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [581.877733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [581.877810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [581.877885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [581.877959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [581.878034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [581.878108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [581.878189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [581.878273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [581.878359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [581.878444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [581.878528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [581.878618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [581.878707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [581.878797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [581.878884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [581.878968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [581.879051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [581.879138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [581.879224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [581.879320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [581.879406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [581.879490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [581.879571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [581.879658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [581.879741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [581.879824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [581.879906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [581.879994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [581.880081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [581.880165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [581.880251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [581.880364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [581.880448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [581.880530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [581.880615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [581.880700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [581.880785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [581.880872] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [581.881463] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [581.881470] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51137, lrc_seqno=51137, guc_id=0, flags=0x73 in no process [-1]
<7> [581.881473] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.881541] ------------[ cut here ]------------
<4> [581.881542] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.881544] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:31/12306
<4> [581.881623] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.881695] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.881700] CPU: 11 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.881703] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.881704] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.881706] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.881712] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.881788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.881790] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [581.881792] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.881794] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.881795] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.881797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.881798] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.881799] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [581.881801] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.881802] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [581.881804] PKRU: 55555554
<4> [581.881805] Call Trace:
<4> [581.881806] <TASK>
<4> [581.881810] ? lock_sync+0x100/0x100
<4> [581.881815] ? lock_release+0xd0/0x2b0
<4> [581.881821] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.881827] process_one_work+0x239/0x760
<4> [581.881834] worker_thread+0x200/0x3f0
<4> [581.881837] ? __pfx_worker_thread+0x10/0x10
<4> [581.881839] kthread+0x10d/0x150
<4> [581.881843] ? __pfx_kthread+0x10/0x10
<4> [581.881846] ret_from_fork+0x3d4/0x480
<4> [581.881849] ? __pfx_kthread+0x10/0x10
<4> [581.881852] ret_from_fork_asm+0x1a/0x30
<4> [581.881860] </TASK>
<4> [581.881861] irq event stamp: 406055
<4> [581.881863] hardirqs last enabled at (406061): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.881866] hardirqs last disabled at (406066): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.881868] softirqs last enabled at (405252): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.881871] softirqs last disabled at (405245): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.881873] ---[ end trace 0000000000000000 ]---
<6> [581.881876] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.881953] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.881959] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.883679] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [581.883790] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [581.883992] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.884204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.884310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.884405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.884495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.884583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.884673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.884763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.884853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.884936] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.885032] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.885148] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.886160] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [581.896885] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [581.897131] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [581.898199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [581.898280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [581.898357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [581.898432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [581.898503] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [581.898573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [581.898643] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [581.898713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [581.898784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [581.898855] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [581.898924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [581.898993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [581.899062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [581.899130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [581.899200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [581.899279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [581.899358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [581.899435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [581.899510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [581.899593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [581.899678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [581.899760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [581.899842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [581.899920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [581.899997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [581.900073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [581.900151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [581.900232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [581.900326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [581.900404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [581.900484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [581.900563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [581.900651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [581.900740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [581.900828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [581.900912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [581.900993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [581.901073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [581.901152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [581.901230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [581.901319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [581.901399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [581.901477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [581.901557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [581.901644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [581.901727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [581.901807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [581.901888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [581.901971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [581.902049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [581.902128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [581.902207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [581.902292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [581.902369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [581.902446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [581.902524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [581.902601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [581.902682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [581.902764] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [581.902878] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [581.902883] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51137, lrc_seqno=51137, guc_id=0, flags=0x73 in no process [-1]
<7> [581.902886] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.902946] ------------[ cut here ]------------
<4> [581.902947] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.902948] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [581.903020] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.903089] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.903094] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.903097] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.903099] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.903100] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.903106] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.903177] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.903179] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [581.903181] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.903182] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.903184] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.903185] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.903186] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.903187] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [581.903189] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.903190] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [581.903192] PKRU: 55555554
<4> [581.903193] Call Trace:
<4> [581.903194] <TASK>
<4> [581.903198] ? lock_sync+0x100/0x100
<4> [581.903203] ? lock_release+0xd0/0x2b0
<4> [581.903209] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.903215] process_one_work+0x239/0x760
<4> [581.903222] worker_thread+0x200/0x3f0
<4> [581.903225] ? __pfx_worker_thread+0x10/0x10
<4> [581.903227] kthread+0x10d/0x150
<4> [581.903230] ? __pfx_kthread+0x10/0x10
<4> [581.903234] ret_from_fork+0x3d4/0x480
<4> [581.903236] ? __pfx_kthread+0x10/0x10
<4> [581.903239] ret_from_fork_asm+0x1a/0x30
<4> [581.903247] </TASK>
<4> [581.903248] irq event stamp: 409181
<4> [581.903249] hardirqs last enabled at (409187): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.903252] hardirqs last disabled at (409192): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.903255] softirqs last enabled at (408110): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.903257] softirqs last disabled at (408103): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.903260] ---[ end trace 0000000000000000 ]---
<5> [581.903956] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51192, lrc_seqno=51192, guc_id=0, flags=0x73 in no process [-1]
<7> [581.903959] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.904019] ------------[ cut here ]------------
<4> [581.904020] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.904021] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [581.904092] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.904158] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.904162] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.904165] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.904166] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.904168] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.904172] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.904241] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.904243] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [581.904245] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.904246] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.904248] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.904249] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.904250] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.904251] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [581.904253] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.904254] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [581.904255] PKRU: 55555554
<4> [581.904257] Call Trace:
<4> [581.904258] <TASK>
<4> [581.904266] ? lock_sync+0x100/0x100
<4> [581.904271] ? lock_release+0xd0/0x2b0
<4> [581.904276] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.904282] process_one_work+0x239/0x760
<4> [581.904288] worker_thread+0x200/0x3f0
<4> [581.904291] ? __pfx_worker_thread+0x10/0x10
<4> [581.904294] kthread+0x10d/0x150
<4> [581.904297] ? __pfx_kthread+0x10/0x10
<4> [581.904301] ret_from_fork+0x3d4/0x480
<4> [581.904303] ? __pfx_kthread+0x10/0x10
<4> [581.904306] ret_from_fork_asm+0x1a/0x30
<4> [581.904313] </TASK>
<4> [581.904314] irq event stamp: 411919
<4> [581.904315] hardirqs last enabled at (411925): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.904318] hardirqs last disabled at (411930): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.904320] softirqs last enabled at (411846): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.904323] softirqs last disabled at (411841): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.904325] ---[ end trace 0000000000000000 ]---
<6> [581.905298] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.905374] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.905383] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.905517] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [581.905623] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [581.905973] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.906182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.906284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.906379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.906468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.906554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.906640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.906725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.906813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.906896] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.906992] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.907110] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.908118] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [581.918269] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [581.918521] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [581.919613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [581.919691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [581.919768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [581.919846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [581.919924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [581.920000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [581.920076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [581.920151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [581.920225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [581.920310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [581.920384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [581.920458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [581.920532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [581.920605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [581.920679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [581.920752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [581.920827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [581.920900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [581.920974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [581.921056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [581.921137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [581.921216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [581.921306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [581.921385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [581.921463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [581.921541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [581.921621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [581.921705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [581.921788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [581.921869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [581.921950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [581.922029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [581.922114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [581.922198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [581.922288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [581.922370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [581.922450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [581.922528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [581.922608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [581.922689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [581.922771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [581.922852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [581.922930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [581.923008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [581.923088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [581.923166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [581.923243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [581.923332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [581.923414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [581.923495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [581.923575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [581.923659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [581.923738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [581.923817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [581.923894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [581.923974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [581.924052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [581.924132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [581.924214] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [581.924334] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [581.924338] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51192, lrc_seqno=51192, guc_id=0, flags=0x73 in no process [-1]
<7> [581.924341] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.924400] ------------[ cut here ]------------
<4> [581.924401] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.924403] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [581.924474] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.924540] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.924545] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.924548] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.924549] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.924550] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.924556] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.924626] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.924628] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [581.924630] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.924632] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.924633] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.924634] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.924635] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.924636] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [581.924638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.924639] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [581.924641] PKRU: 55555554
<4> [581.924642] Call Trace:
<4> [581.924643] <TASK>
<4> [581.924647] ? lock_sync+0x100/0x100
<4> [581.924652] ? lock_release+0xd0/0x2b0
<4> [581.924657] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.924663] process_one_work+0x239/0x760
<4> [581.924669] worker_thread+0x200/0x3f0
<4> [581.924672] ? __pfx_worker_thread+0x10/0x10
<4> [581.924674] kthread+0x10d/0x150
<4> [581.924678] ? __pfx_kthread+0x10/0x10
<4> [581.924681] ret_from_fork+0x3d4/0x480
<4> [581.924683] ? __pfx_kthread+0x10/0x10
<4> [581.924687] ret_from_fork_asm+0x1a/0x30
<4> [581.924694] </TASK>
<4> [581.924695] irq event stamp: 415047
<4> [581.924696] hardirqs last enabled at (415053): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.924699] hardirqs last disabled at (415058): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.924702] softirqs last enabled at (414028): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.924704] softirqs last disabled at (414021): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.924707] ---[ end trace 0000000000000000 ]---
<6> [581.924708] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.924779] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.924785] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.925064] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.925271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.925361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.925455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.925544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.925630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.925714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.925804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.925894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.925979] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.926073] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.926189] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.927197] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [581.937925] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [581.938172] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [581.939390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [581.939461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [581.939532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [581.939603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [581.939673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [581.939742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [581.939811] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [581.939879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [581.939946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [581.940014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [581.940082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [581.940150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [581.940221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [581.940305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [581.940379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [581.940447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [581.940516] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [581.940583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [581.940649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [581.940724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [581.940799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [581.940873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [581.940947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [581.941018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [581.941088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [581.941158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [581.941228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [581.941309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [581.941381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [581.941453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [581.941526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [581.941602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [581.941681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [581.941759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [581.941838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [581.941916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [581.941992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [581.942064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [581.942136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [581.942205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [581.942281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [581.942352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [581.942425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [581.942499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [581.942574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [581.942645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [581.942716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [581.942786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [581.942857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [581.942931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [581.943003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [581.943076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [581.943148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [581.943219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [581.943295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [581.943366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [581.943435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [581.943507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [581.943583] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [581.943688] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [581.943691] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51192, lrc_seqno=51192, guc_id=0, flags=0x73 in no process [-1]
<7> [581.943694] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.943747] ------------[ cut here ]------------
<4> [581.943748] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.943749] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [581.943814] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.943874] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.943878] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.943881] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.943882] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.943883] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.943888] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.943951] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.943952] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [581.943954] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.943956] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.943957] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.943958] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.943959] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.943960] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [581.943962] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.943963] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [581.943964] PKRU: 55555554
<4> [581.943965] Call Trace:
<4> [581.943966] <TASK>
<4> [581.943970] ? lock_sync+0x100/0x100
<4> [581.943974] ? lock_release+0xd0/0x2b0
<4> [581.943979] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.943984] process_one_work+0x239/0x760
<4> [581.943990] worker_thread+0x200/0x3f0
<4> [581.943993] ? __pfx_worker_thread+0x10/0x10
<4> [581.943995] kthread+0x10d/0x150
<4> [581.943998] ? __pfx_kthread+0x10/0x10
<4> [581.944001] ret_from_fork+0x3d4/0x480
<4> [581.944003] ? __pfx_kthread+0x10/0x10
<4> [581.944006] ret_from_fork_asm+0x1a/0x30
<4> [581.944012] </TASK>
<4> [581.944013] irq event stamp: 418179
<4> [581.944015] hardirqs last enabled at (418185): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.944017] hardirqs last disabled at (418190): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.944019] softirqs last enabled at (417306): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.944022] softirqs last disabled at (417301): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.944024] ---[ end trace 0000000000000000 ]---
<7> [581.945518] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [581.945623] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [581.945977] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51247, lrc_seqno=51247, guc_id=0, flags=0x73 in no process [-1]
<7> [581.945982] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.946055] ------------[ cut here ]------------
<4> [581.946056] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.946058] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [581.946138] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.946204] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.946209] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.946212] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.946213] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.946215] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.946220] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.946305] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.946307] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [581.946310] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.946312] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.946313] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.946314] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.946316] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.946317] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [581.946319] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.946321] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [581.946322] PKRU: 55555554
<4> [581.946324] Call Trace:
<4> [581.946325] <TASK>
<4> [581.946329] ? lock_sync+0x100/0x100
<4> [581.946334] ? lock_release+0xd0/0x2b0
<4> [581.946340] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.946346] process_one_work+0x239/0x760
<4> [581.946352] worker_thread+0x200/0x3f0
<4> [581.946355] ? __pfx_worker_thread+0x10/0x10
<4> [581.946358] kthread+0x10d/0x150
<4> [581.946361] ? __pfx_kthread+0x10/0x10
<4> [581.946365] ret_from_fork+0x3d4/0x480
<4> [581.946367] ? __pfx_kthread+0x10/0x10
<4> [581.946370] ret_from_fork_asm+0x1a/0x30
<4> [581.946378] </TASK>
<4> [581.946379] irq event stamp: 101353
<4> [581.946381] hardirqs last enabled at (101359): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.946384] hardirqs last disabled at (101364): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.946386] softirqs last enabled at (98686): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [581.946388] softirqs last disabled at (98684): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [581.946391] ---[ end trace 0000000000000000 ]---
<6> [581.946393] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.946468] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.946475] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.946675] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.946885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.946983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.947082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.947176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.947276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.947371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.947463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.947556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.947644] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.947745] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.947867] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.948872] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [581.959287] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [581.959536] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [581.960632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [581.960751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [581.960824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [581.960896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [581.960966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [581.961035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [581.961104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [581.961172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [581.961240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [581.961321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [581.961390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [581.961457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [581.961525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [581.961594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [581.961662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [581.961734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [581.961805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [581.961874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [581.961943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [581.962018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [581.962091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [581.962164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [581.962237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [581.962317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [581.962387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [581.962458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [581.962536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [581.962612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [581.962689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [581.962763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [581.962836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [581.962910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [581.962988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [581.963065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [581.963143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [581.963216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [581.963297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [581.963368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [581.963442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [581.963517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [581.963592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [581.963665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [581.963737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [581.963807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [581.963881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [581.963951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [581.964022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [581.964091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [581.964163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [581.964233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [581.964312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [581.964386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [581.964456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [581.964526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [581.964595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [581.964666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [581.964735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [581.964807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [581.964885] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [581.964990] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [581.964994] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51247, lrc_seqno=51247, guc_id=0, flags=0x73 in no process [-1]
<7> [581.964996] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.965050] ------------[ cut here ]------------
<4> [581.965051] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.965052] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [581.965117] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.965177] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.965181] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.965184] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.965185] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.965186] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.965191] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.965254] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.965255] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [581.965257] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.965259] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.965260] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.965266] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.965267] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.965269] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [581.965270] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.965272] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [581.965273] PKRU: 55555554
<4> [581.965274] Call Trace:
<4> [581.965275] <TASK>
<4> [581.965279] ? lock_sync+0x100/0x100
<4> [581.965283] ? lock_release+0xd0/0x2b0
<4> [581.965289] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.965294] process_one_work+0x239/0x760
<4> [581.965300] worker_thread+0x200/0x3f0
<4> [581.965303] ? __pfx_worker_thread+0x10/0x10
<4> [581.965305] kthread+0x10d/0x150
<4> [581.965307] ? __pfx_kthread+0x10/0x10
<4> [581.965311] ret_from_fork+0x3d4/0x480
<4> [581.965313] ? __pfx_kthread+0x10/0x10
<4> [581.965316] ret_from_fork_asm+0x1a/0x30
<4> [581.965323] </TASK>
<4> [581.965324] irq event stamp: 104595
<4> [581.965325] hardirqs last enabled at (104601): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.965327] hardirqs last disabled at (104606): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.965329] softirqs last enabled at (103692): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.965332] softirqs last disabled at (103687): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.965334] ---[ end trace 0000000000000000 ]---
<6> [581.966275] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.966344] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.966354] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.966368] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [581.966463] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [581.966563] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.966823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.966960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.967099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.967231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.967372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.967503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.967636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.967770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.967900] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.968042] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.968207] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.969605] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [581.980273] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [581.980580] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [581.981782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [581.981903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [581.982021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [581.982141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [581.982261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [581.982387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [581.982509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [581.982629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [581.982749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [581.982869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [581.982989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [581.983109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [581.983228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [581.983354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [581.983477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [581.983596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [581.983716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [581.983836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [581.983956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [581.984080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [581.984205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [581.984334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [581.984461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [581.984584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [581.984707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [581.984830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [581.984952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [581.985079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [581.985206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [581.985335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [581.985462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [581.985588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [581.985719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [581.985850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [581.985980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [581.986108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [581.986237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [581.986365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [581.986491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [581.986613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [581.986741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [581.986866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [581.986990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [581.987113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [581.987242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [581.987370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [581.987495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [581.987619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [581.987744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [581.987867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [581.987990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [581.988128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [581.988217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [581.988313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [581.988389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [581.988464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [581.988536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [581.988606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [581.988680] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [581.988787] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [581.988792] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51247, lrc_seqno=51247, guc_id=0, flags=0x73 in no process [-1]
<7> [581.988794] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.988847] ------------[ cut here ]------------
<4> [581.988848] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.988850] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:41/12316
<4> [581.988915] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.988977] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.988982] CPU: 0 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.988984] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.988986] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.988987] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.988992] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.989055] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.989057] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [581.989059] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.989061] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.989062] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.989063] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.989064] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.989065] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [581.989067] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.989068] CR2: 00007ca99c096388 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [581.989070] PKRU: 55555554
<4> [581.989071] Call Trace:
<4> [581.989072] <TASK>
<4> [581.989075] ? lock_sync+0x100/0x100
<4> [581.989080] ? lock_release+0xd0/0x2b0
<4> [581.989085] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.989090] process_one_work+0x239/0x760
<4> [581.989096] worker_thread+0x200/0x3f0
<4> [581.989099] ? __pfx_worker_thread+0x10/0x10
<4> [581.989101] kthread+0x10d/0x150
<4> [581.989103] ? __pfx_kthread+0x10/0x10
<4> [581.989107] ret_from_fork+0x3d4/0x480
<4> [581.989109] ? __pfx_kthread+0x10/0x10
<4> [581.989112] ret_from_fork_asm+0x1a/0x30
<4> [581.989118] </TASK>
<4> [581.989119] irq event stamp: 138741
<4> [581.989120] hardirqs last enabled at (138747): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.989123] hardirqs last disabled at (138752): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.989125] softirqs last enabled at (137862): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.989128] softirqs last disabled at (137857): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.989130] ---[ end trace 0000000000000000 ]---
<5> [581.990551] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51302, lrc_seqno=51302, guc_id=0, flags=0x73 in no process [-1]
<7> [581.990555] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [581.990628] ------------[ cut here ]------------
<4> [581.990630] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [581.990632] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:41/12316
<4> [581.990708] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [581.990774] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [581.990778] CPU: 0 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [581.990781] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [581.990782] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [581.990784] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [581.990789] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [581.990857] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [581.990859] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [581.990861] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [581.990862] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [581.990864] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [581.990865] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [581.990866] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [581.990867] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [581.990869] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [581.990870] CR2: 00007ca99c096388 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [581.990871] PKRU: 55555554
<4> [581.990873] Call Trace:
<4> [581.990874] <TASK>
<4> [581.990877] ? lock_sync+0x100/0x100
<4> [581.990882] ? lock_release+0xd0/0x2b0
<4> [581.990887] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [581.990893] process_one_work+0x239/0x760
<4> [581.990899] worker_thread+0x200/0x3f0
<4> [581.990901] ? __pfx_worker_thread+0x10/0x10
<4> [581.990904] kthread+0x10d/0x150
<4> [581.990906] ? __pfx_kthread+0x10/0x10
<4> [581.990910] ret_from_fork+0x3d4/0x480
<4> [581.990912] ? __pfx_kthread+0x10/0x10
<4> [581.990915] ret_from_fork_asm+0x1a/0x30
<4> [581.990923] </TASK>
<4> [581.990924] irq event stamp: 141399
<4> [581.990925] hardirqs last enabled at (141405): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [581.990927] hardirqs last disabled at (141410): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [581.990930] softirqs last enabled at (137862): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.990932] softirqs last disabled at (137857): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [581.990934] ---[ end trace 0000000000000000 ]---
<6> [581.990936] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [581.991005] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [581.991011] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [581.991226] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [581.991347] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [581.991454] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [581.991659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [581.991747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [581.991842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [581.991932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [581.992021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [581.992107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [581.992192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [581.992285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [581.992368] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [581.992463] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [581.992580] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [581.993597] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.004274] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.004529] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.005890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.005978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.006057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.006133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.006208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.006294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.006379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.006459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.006538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.006618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.006696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.006774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.006857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.006938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.007018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.007098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.007177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.007257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.007353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.007440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.007527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.007611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.007699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.007786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.007871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.007953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.008035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.008120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.008205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.008295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.008379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.008463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.008557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.008648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.008738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.008824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.008908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.008991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.009078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.009162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.009250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.009345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.009429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.009511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.009596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.009682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.009767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.009851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.009937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.010022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.010106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.010192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.010278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.010362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.010443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.010525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.010609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.010694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.010781] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.010898] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.010903] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51302, lrc_seqno=51302, guc_id=0, flags=0x73 in no process [-1]
<7> [582.010905] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.010967] ------------[ cut here ]------------
<4> [582.010969] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.010970] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:41/12316
<4> [582.011046] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.011111] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.011116] CPU: 4 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.011119] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.011120] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.011122] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.011127] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.011202] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.011204] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.011206] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.011208] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.011209] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.011210] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.011212] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.011213] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.011215] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.011216] CR2: 00007ddcc54a0008 CR3: 00000001acb3b006 CR4: 0000000000f72ef0
<4> [582.011218] PKRU: 55555554
<4> [582.011219] Call Trace:
<4> [582.011220] <TASK>
<4> [582.011224] ? lock_sync+0x100/0x100
<4> [582.011229] ? lock_release+0xd0/0x2b0
<4> [582.011235] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.011240] process_one_work+0x239/0x760
<4> [582.011247] worker_thread+0x200/0x3f0
<4> [582.011250] ? __pfx_worker_thread+0x10/0x10
<4> [582.011252] kthread+0x10d/0x150
<4> [582.011255] ? __pfx_kthread+0x10/0x10
<4> [582.011259] ret_from_fork+0x3d4/0x480
<4> [582.011261] ? __pfx_kthread+0x10/0x10
<4> [582.011268] ret_from_fork_asm+0x1a/0x30
<4> [582.011276] </TASK>
<4> [582.011277] irq event stamp: 144535
<4> [582.011279] hardirqs last enabled at (144541): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.011282] hardirqs last disabled at (144546): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.011284] softirqs last enabled at (144518): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.011287] softirqs last disabled at (144513): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.011289] ---[ end trace 0000000000000000 ]---
<6> [582.011291] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.011366] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.011372] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.012672] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.012813] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.012939] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.013144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.013233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.013336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.013424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.013511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.013590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.013667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.013746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.013820] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.013907] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.014015] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.015013] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.025619] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.025860] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.026914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.026994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.027071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.027148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.027224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.027309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.027385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.027461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.027538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.027613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.027689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.027763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.027838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.027914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.027989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.028067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.028145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.028222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.028307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.028390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.028473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.028553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.028634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.028713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.028791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.028869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.028946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.029031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.029116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.029198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.029282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.029362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.029447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.029532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.029617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.029698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.029778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.029856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.029933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.030010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.030090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.030169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.030251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.030343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.030427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.030504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.030583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.030660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.030739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.030815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.030891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.030969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.031046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.031124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.031207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.031293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.031372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.031450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.031530] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.031642] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.031646] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51302, lrc_seqno=51302, guc_id=0, flags=0x73 in no process [-1]
<7> [582.031648] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.031706] ------------[ cut here ]------------
<4> [582.031708] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.031709] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.031781] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.031846] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.031850] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.031853] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.031854] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.031856] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.031861] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.031931] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.031933] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.031935] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.031937] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.031938] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.031939] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.031941] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.031942] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.031944] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.031945] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.031947] PKRU: 55555554
<4> [582.031948] Call Trace:
<4> [582.031949] <TASK>
<4> [582.031953] ? lock_sync+0x100/0x100
<4> [582.031957] ? lock_release+0xd0/0x2b0
<4> [582.031963] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.031969] process_one_work+0x239/0x760
<4> [582.031975] worker_thread+0x200/0x3f0
<4> [582.031977] ? __pfx_worker_thread+0x10/0x10
<4> [582.031980] kthread+0x10d/0x150
<4> [582.031983] ? __pfx_kthread+0x10/0x10
<4> [582.031986] ret_from_fork+0x3d4/0x480
<4> [582.031988] ? __pfx_kthread+0x10/0x10
<4> [582.031992] ret_from_fork_asm+0x1a/0x30
<4> [582.031999] </TASK>
<4> [582.032000] irq event stamp: 422349
<4> [582.032001] hardirqs last enabled at (422355): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.032004] hardirqs last disabled at (422360): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.032007] softirqs last enabled at (421294): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.032009] softirqs last disabled at (421287): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.032011] ---[ end trace 0000000000000000 ]---
<5> [582.033521] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51357, lrc_seqno=51357, guc_id=0, flags=0x73 in no process [-1]
<7> [582.033525] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [582.033530] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [582.033591] ------------[ cut here ]------------
<4> [582.033592] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.033593] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [582.033668] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers
<7> [582.033636] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [582.033685] i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.033739] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.033742] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.033743] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.033745] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.033750] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.033822] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.033824] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.033827] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.033828] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.033829] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.033831] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.033832] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.033833] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [582.033835] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.033836] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.033838] PKRU: 55555554
<4> [582.033839] Call Trace:
<4> [582.033840] <TASK>
<4> [582.033844] ? lock_sync+0x100/0x100
<4> [582.033849] ? lock_release+0xd0/0x2b0
<4> [582.033854] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.033860] process_one_work+0x239/0x760
<4> [582.033866] worker_thread+0x200/0x3f0
<4> [582.033869] ? __pfx_worker_thread+0x10/0x10
<4> [582.033871] kthread+0x10d/0x150
<4> [582.033874] ? __pfx_kthread+0x10/0x10
<4> [582.033877] ret_from_fork+0x3d4/0x480
<4> [582.033880] ? __pfx_kthread+0x10/0x10
<4> [582.033883] ret_from_fork_asm+0x1a/0x30
<4> [582.033890] </TASK>
<4> [582.033891] irq event stamp: 108421
<4> [582.033893] hardirqs last enabled at (108427): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.033895] hardirqs last disabled at (108432): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.033898] softirqs last enabled at (106690): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [582.033900] softirqs last disabled at (106688): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [582.033902] ---[ end trace 0000000000000000 ]---
<6> [582.033904] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.033976] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.033982] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.034160] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.034369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.034460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.034553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.034642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.034729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.034814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.034900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.034988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.035072] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.035169] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.035289] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.036297] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.046304] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.046554] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.047627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.047707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.047784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.047862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.047939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.048017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.048094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.048170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.048246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.048331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.048406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.048481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.048556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.048630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.048705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.048778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.048852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.048926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.049000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.049080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.049162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.049243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.049334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.049412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.049490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.049568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.049645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.049724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.049805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.049884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.049963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.050042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.050131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.050218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.050312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.050394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.050475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.050553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.050631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.050707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.050791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.050874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.050955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.051034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.051119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.051202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.051285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.051364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.051444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.051522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.051602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.051686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.051766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.051845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.051923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.052001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.052079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.052156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.052237] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.052360] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.052364] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51357, lrc_seqno=51357, guc_id=0, flags=0x73 in no process [-1]
<7> [582.052366] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.052424] ------------[ cut here ]------------
<4> [582.052425] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.052427] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.052498] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.052562] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.052567] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.052570] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.052571] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.052573] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.052578] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.052648] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.052650] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.052652] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.052653] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.052655] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.052656] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.052657] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.052658] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.052660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.052661] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.052663] PKRU: 55555554
<4> [582.052664] Call Trace:
<4> [582.052665] <TASK>
<4> [582.052669] ? lock_sync+0x100/0x100
<4> [582.052673] ? lock_release+0xd0/0x2b0
<4> [582.052679] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.052684] process_one_work+0x239/0x760
<4> [582.052690] worker_thread+0x200/0x3f0
<4> [582.052693] ? __pfx_worker_thread+0x10/0x10
<4> [582.052695] kthread+0x10d/0x150
<4> [582.052698] ? __pfx_kthread+0x10/0x10
<4> [582.052702] ret_from_fork+0x3d4/0x480
<4> [582.052704] ? __pfx_kthread+0x10/0x10
<4> [582.052707] ret_from_fork_asm+0x1a/0x30
<4> [582.052715] </TASK>
<4> [582.052716] irq event stamp: 426939
<4> [582.052717] hardirqs last enabled at (426945): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.052720] hardirqs last disabled at (426950): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.052722] softirqs last enabled at (425920): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.052725] softirqs last disabled at (425913): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.052727] ---[ end trace 0000000000000000 ]---
<6> [582.052729] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.052798] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.052804] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.053211] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.053619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.053704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.053796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.053886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.053973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.054059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.054145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.054232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.054326] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.054426] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.054544] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.055564] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.066285] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.066534] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.067594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.067676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.067753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.067831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.067908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.067984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.068059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.068133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.068208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.068291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.068370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.068447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.068524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.068599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.068675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.068751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.068824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.068898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.068971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.069052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.069133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.069213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.069302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.069382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.069461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.069538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.069614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.069693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.069775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.069853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.069936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.070019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.070104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.070188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.070277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.070362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.070445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.070525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.070603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.070681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.070761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.070840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.070918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.070994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.071079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.071160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.071239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.071325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.071403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.071478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.071555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.071633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.071710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.071786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.071862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.071940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.072015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.072091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.072170] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.072286] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.072290] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51357, lrc_seqno=51357, guc_id=0, flags=0x73 in no process [-1]
<7> [582.072292] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.072350] ------------[ cut here ]------------
<4> [582.072351] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.072353] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.072426] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.072490] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.072494] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.072497] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.072498] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.072499] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.072504] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.072575] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.072577] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.072579] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.072580] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.072582] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.072583] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.072584] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.072585] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.072587] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.072588] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.072590] PKRU: 55555554
<4> [582.072591] Call Trace:
<4> [582.072592] <TASK>
<4> [582.072596] ? lock_sync+0x100/0x100
<4> [582.072600] ? lock_release+0xd0/0x2b0
<4> [582.072606] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.072611] process_one_work+0x239/0x760
<4> [582.072617] worker_thread+0x200/0x3f0
<4> [582.072620] ? __pfx_worker_thread+0x10/0x10
<4> [582.072622] kthread+0x10d/0x150
<4> [582.072625] ? __pfx_kthread+0x10/0x10
<4> [582.072629] ret_from_fork+0x3d4/0x480
<4> [582.072631] ? __pfx_kthread+0x10/0x10
<4> [582.072634] ret_from_fork_asm+0x1a/0x30
<4> [582.072641] </TASK>
<4> [582.072642] irq event stamp: 430071
<4> [582.072643] hardirqs last enabled at (430077): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.072646] hardirqs last disabled at (430082): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.072648] softirqs last enabled at (429264): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.072651] softirqs last disabled at (429257): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.072653] ---[ end trace 0000000000000000 ]---
<5> [582.074145] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51412, lrc_seqno=51412, guc_id=0, flags=0x73 in no process [-1]
<7> [582.074148] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.074208] ------------[ cut here ]------------
<4> [582.074209] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.074211] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.074291] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.074355] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.074359] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.074362] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.074363] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.074364] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.074369] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.074438] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.074439] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.074441] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.074442] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.074444] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.074445] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.074446] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.074447] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.074449] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.074450] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.074451] PKRU: 55555554
<4> [582.074453] Call Trace:
<4> [582.074454] <TASK>
<4> [582.074457] ? lock_sync+0x100/0x100
<4> [582.074461] ? lock_release+0xd0/0x2b0
<4> [582.074467] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.074472] process_one_work+0x239/0x760
<4> [582.074478] worker_thread+0x200/0x3f0
<4> [582.074481] ? __pfx_worker_thread+0x10/0x10
<4> [582.074483] kthread+0x10d/0x150
<4> [582.074486] ? __pfx_kthread+0x10/0x10
<4> [582.074489] ret_from_fork+0x3d4/0x480
<4> [582.074491] ? __pfx_kthread+0x10/0x10
<4> [582.074495] ret_from_fork_asm+0x1a/0x30
<4> [582.074502] </TASK>
<4> [582.074503] irq event stamp: 432741
<4> [582.074504] hardirqs last enabled at (432747): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.074507] hardirqs last disabled at (432752): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.074509] softirqs last enabled at (431992): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.074511] softirqs last disabled at (431981): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.074513] ---[ end trace 0000000000000000 ]---
<7> [582.074527] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.074633] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [582.074731] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.074803] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.074809] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.074858] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.075064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.075156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.075252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.075364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.075480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.075608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.075698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.075788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.075872] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.075969] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.076086] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.077097] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.087274] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.087529] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.088609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.088690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.088768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.088845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.088922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.088999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.089075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.089149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.089222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.089309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.089383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.089457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.089531] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.089605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.089678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.089755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.089833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.089910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.089985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.090067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.090149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.090228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.090319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.090398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.090476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.090554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.090637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.090721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.090805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.090887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.090967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.091047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.091133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.091279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.091386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.091474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.091557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.091635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.091713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.091790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.091870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.091949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.092026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.092102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.092185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.092273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.092354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.092432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.092512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.092588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.092664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.092745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.092822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.092904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.092984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.093063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.093139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.093215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.093302] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.093415] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.093419] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51412, lrc_seqno=51412, guc_id=0, flags=0x73 in no process [-1]
<7> [582.093421] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.093479] ------------[ cut here ]------------
<4> [582.093481] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.093482] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [582.093554] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.093619] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.093623] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.093626] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.093628] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.093629] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.093634] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.093704] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.093706] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.093708] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.093710] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.093711] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.093712] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.093713] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.093715] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [582.093716] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.093718] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.093719] PKRU: 55555554
<4> [582.093721] Call Trace:
<4> [582.093722] <TASK>
<4> [582.093726] ? lock_sync+0x100/0x100
<4> [582.093730] ? lock_release+0xd0/0x2b0
<4> [582.093736] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.093741] process_one_work+0x239/0x760
<4> [582.093747] worker_thread+0x200/0x3f0
<4> [582.093750] ? __pfx_worker_thread+0x10/0x10
<4> [582.093752] kthread+0x10d/0x150
<4> [582.093755] ? __pfx_kthread+0x10/0x10
<4> [582.093759] ret_from_fork+0x3d4/0x480
<4> [582.093761] ? __pfx_kthread+0x10/0x10
<4> [582.093764] ret_from_fork_asm+0x1a/0x30
<4> [582.093772] </TASK>
<4> [582.093773] irq event stamp: 113803
<4> [582.093774] hardirqs last enabled at (113809): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.093777] hardirqs last disabled at (113814): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.093779] softirqs last enabled at (112954): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.093781] softirqs last disabled at (112947): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.093784] ---[ end trace 0000000000000000 ]---
<6> [582.093785] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.093855] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.093861] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.094279] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.094482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.094570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.094661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.094749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.094835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.094921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.095008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.095097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.095180] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.095282] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.095400] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.096408] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.107131] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.107382] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.108453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.108534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.108612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.108690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.108769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.108848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.108926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.109003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.109080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.109156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.109231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.109322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.109395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.109470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.109545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.109619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.109693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.109767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.109845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.109928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.110012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.110092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.110172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.110250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.110340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.110419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.110497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.110582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.110667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.110748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.110836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.110916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.111002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.111088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.111174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.111256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.111354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.111436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.111518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.111597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.111678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.111755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.111833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.111910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.111991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.112067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.112144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.112224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.112317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.112395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.112473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.112554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.112633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.112712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.112794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.112876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.112955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.113033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.113114] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.113226] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.113230] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51412, lrc_seqno=51412, guc_id=0, flags=0x73 in no process [-1]
<7> [582.113232] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.114441] ------------[ cut here ]------------
<4> [582.114442] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.114444] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [582.114518] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.114583] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.114588] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.114591] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.114592] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.114593] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.114598] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.114669] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.114671] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.114673] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.114675] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.114676] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.114677] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.114679] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.114680] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [582.114682] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.114683] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.114684] PKRU: 55555554
<4> [582.114686] Call Trace:
<4> [582.114687] <TASK>
<4> [582.114691] ? lock_sync+0x100/0x100
<4> [582.114695] ? lock_release+0xd0/0x2b0
<4> [582.114701] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.114707] process_one_work+0x239/0x760
<4> [582.114713] worker_thread+0x200/0x3f0
<4> [582.114715] ? __pfx_worker_thread+0x10/0x10
<4> [582.114718] kthread+0x10d/0x150
<4> [582.114721] ? __pfx_kthread+0x10/0x10
<4> [582.114724] ret_from_fork+0x3d4/0x480
<4> [582.114726] ? __pfx_kthread+0x10/0x10
<4> [582.114730] ret_from_fork_asm+0x1a/0x30
<4> [582.114737] </TASK>
<4> [582.114738] irq event stamp: 116921
<4> [582.114739] hardirqs last enabled at (116927): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.114742] hardirqs last disabled at (116932): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.114744] softirqs last enabled at (116150): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.114747] softirqs last disabled at (116145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.114749] ---[ end trace 0000000000000000 ]---
<7> [582.115298] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<5> [582.115381] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51467, lrc_seqno=51467, guc_id=0, flags=0x73 in no process [-1]
<7> [582.115387] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.115489] ------------[ cut here ]------------
<4> [582.115491] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [582.115446] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [582.115493] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.115606] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.115672] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.115677] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.115679] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.115681] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.115682] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.115687] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.115759] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.115761] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.115763] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.115764] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.115765] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.115766] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.115768] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.115769] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.115771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.115772] CR2: 00007bda80284058 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.115773] PKRU: 55555554
<4> [582.115775] Call Trace:
<4> [582.115776] <TASK>
<4> [582.115780] ? lock_sync+0x100/0x100
<4> [582.115785] ? lock_release+0xd0/0x2b0
<4> [582.115790] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.115796] process_one_work+0x239/0x760
<4> [582.115802] worker_thread+0x200/0x3f0
<4> [582.115805] ? __pfx_worker_thread+0x10/0x10
<4> [582.115807] kthread+0x10d/0x150
<4> [582.115810] ? __pfx_kthread+0x10/0x10
<4> [582.115814] ret_from_fork+0x3d4/0x480
<4> [582.115816] ? __pfx_kthread+0x10/0x10
<4> [582.115819] ret_from_fork_asm+0x1a/0x30
<4> [582.115826] </TASK>
<4> [582.115827] irq event stamp: 119449
<4> [582.115829] hardirqs last enabled at (119455): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.115831] hardirqs last disabled at (119460): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.115833] softirqs last enabled at (116150): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.115836] softirqs last disabled at (116145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.115838] ---[ end trace 0000000000000000 ]---
<6> [582.115840] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.115913] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.115920] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.116036] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.116245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.116349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.116444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.116532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.116620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.116706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.116792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.116881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.116964] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.117059] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.117178] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.118183] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.128273] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.128523] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.129607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.129686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.129762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.129841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.129918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.129995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.130072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.130148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.130222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.130310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.130386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.130460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.130535] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.130610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.130685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.130759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.130834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.130908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.130983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.131066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.131149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.131230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.131326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.131404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.131481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.131559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.131636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.131716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.131801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.131883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.131965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.132046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.132132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.132217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.132313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.132394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.132475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.132553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.132631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.132712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.132794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.132874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.132952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.133029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.133115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.133195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.133279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.133356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.133435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.133510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.133586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.133665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.133741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.133817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.133896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.133976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.134054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.134130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.134209] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.134329] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.134334] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51467, lrc_seqno=51467, guc_id=0, flags=0x73 in no process [-1]
<7> [582.134337] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.134396] ------------[ cut here ]------------
<4> [582.134397] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.134398] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:41/12316
<4> [582.134469] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.134534] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.134539] CPU: 0 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.134542] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.134543] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.134544] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.134549] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.134617] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.134619] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.134621] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.134623] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.134624] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.134626] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.134627] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.134628] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.134630] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.134631] CR2: 00007ca99c096388 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [582.134633] PKRU: 55555554
<4> [582.134634] Call Trace:
<4> [582.134635] <TASK>
<4> [582.134639] ? lock_sync+0x100/0x100
<4> [582.134643] ? lock_release+0xd0/0x2b0
<4> [582.134649] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.134654] process_one_work+0x239/0x760
<4> [582.134661] worker_thread+0x200/0x3f0
<4> [582.134663] ? __pfx_worker_thread+0x10/0x10
<4> [582.134666] kthread+0x10d/0x150
<4> [582.134669] ? __pfx_kthread+0x10/0x10
<4> [582.134672] ret_from_fork+0x3d4/0x480
<4> [582.134674] ? __pfx_kthread+0x10/0x10
<4> [582.134678] ret_from_fork_asm+0x1a/0x30
<4> [582.134685] </TASK>
<4> [582.134686] irq event stamp: 151881
<4> [582.134687] hardirqs last enabled at (151887): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.134690] hardirqs last disabled at (151892): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.134692] softirqs last enabled at (151040): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.134695] softirqs last disabled at (151029): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.134697] ---[ end trace 0000000000000000 ]---
<6> [582.134699] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.134769] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.134775] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.135180] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.135384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.135474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.135567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.135655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.135742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.135827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.135912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.136000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.136083] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.136180] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.136305] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.137329] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.147273] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.147525] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.148604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.148686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.148763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.148840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.148915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.148991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.149066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.149141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.149216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.149305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.149383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.149461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.149537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.149612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.149686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.149759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.149833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.149906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.149979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.150061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.150142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.150225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.150316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.150394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.150471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.150548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.150626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.150706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.150786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.150863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.150940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.151019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.151107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.151195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.151289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.151371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.151452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.151531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.151607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.151684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.151765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.151843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.151925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.152004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.152086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.152163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.152241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.152331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.152409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.152485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.152562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.152640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.152717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.152797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.152877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.152957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.153035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.153112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.153192] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.153308] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.153312] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51467, lrc_seqno=51467, guc_id=0, flags=0x73 in no process [-1]
<7> [582.153315] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.153374] ------------[ cut here ]------------
<4> [582.153375] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.153376] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:41/12316
<4> [582.153447] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.153511] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.153516] CPU: 0 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.153519] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.153520] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.153521] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.153526] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.153595] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.153597] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.153599] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.153601] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.153602] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.153603] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.153605] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.153606] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.153607] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.153609] CR2: 00007ca99c096388 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.153610] PKRU: 55555554
<4> [582.153611] Call Trace:
<4> [582.153613] <TASK>
<4> [582.153616] ? lock_sync+0x100/0x100
<4> [582.153621] ? lock_release+0xd0/0x2b0
<4> [582.153626] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.153632] process_one_work+0x239/0x760
<4> [582.153638] worker_thread+0x200/0x3f0
<4> [582.153641] ? __pfx_worker_thread+0x10/0x10
<4> [582.153643] kthread+0x10d/0x150
<4> [582.153646] ? __pfx_kthread+0x10/0x10
<4> [582.153649] ret_from_fork+0x3d4/0x480
<4> [582.153652] ? __pfx_kthread+0x10/0x10
<4> [582.153655] ret_from_fork_asm+0x1a/0x30
<4> [582.153662] </TASK>
<4> [582.153663] irq event stamp: 154985
<4> [582.153665] hardirqs last enabled at (154991): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.153667] hardirqs last disabled at (154996): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.153670] softirqs last enabled at (153966): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.153672] softirqs last disabled at (153959): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.153674] ---[ end trace 0000000000000000 ]---
<5> [582.155174] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51522, lrc_seqno=51522, guc_id=0, flags=0x73 in no process [-1]
<7> [582.155177] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.155237] ------------[ cut here ]------------
<4> [582.155238] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.155240] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:41/12316
<4> [582.155323] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.155388] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.155392] CPU: 0 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.155395] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.155396] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.155397] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.155402] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.155472] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.155474] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.155476] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.155477] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.155478] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.155480] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.155481] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.155482] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.155484] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.155485] CR2: 00007ca99c096388 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.155486] PKRU: 55555554
<4> [582.155487] Call Trace:
<4> [582.155488] <TASK>
<4> [582.155492] ? lock_sync+0x100/0x100
<4> [582.155496] ? lock_release+0xd0/0x2b0
<4> [582.155502] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.155507] process_one_work+0x239/0x760
<4> [582.155513] worker_thread+0x200/0x3f0
<4> [582.155516] ? __pfx_worker_thread+0x10/0x10
<4> [582.155518] kthread+0x10d/0x150
<4> [582.155521] ? __pfx_kthread+0x10/0x10
<4> [582.155525] ret_from_fork+0x3d4/0x480
<4> [582.155527] ? __pfx_kthread+0x10/0x10
<4> [582.155530] ret_from_fork_asm+0x1a/0x30
<4> [582.155537] </TASK>
<4> [582.155538] irq event stamp: 157691
<4> [582.155539] hardirqs last enabled at (157697): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.155542] hardirqs last disabled at (157702): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.155544] softirqs last enabled at (156942): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.155547] softirqs last disabled at (156935): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.155549] ---[ end trace 0000000000000000 ]---
<7> [582.155563] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.155668] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [582.155765] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.155836] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.155842] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.155891] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.156100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.156191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.156289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.156381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.156470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.156556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.156643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.156733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.156821] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.156918] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.157035] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.158043] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.168274] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.168523] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.169614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.169693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.169769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.169848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.169925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.170002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.170076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.170151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.170225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.170307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.170381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.170455] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.170529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.170603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.170677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.170750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.170824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.170899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.170972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.171053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.171136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.171218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.171308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.171386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.171463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.171541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.171618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.171701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.171784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.171864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.171945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.172025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.172110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.172195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.172282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.172365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.172445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.172523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.172600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.172677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.172761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.172843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.172922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.173001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.173083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.173161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.173239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.173324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.173403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.173480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.173561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.173645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.173726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.173807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.173886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.173965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.174043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.174121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.174201] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.174318] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.174322] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51522, lrc_seqno=51522, guc_id=0, flags=0x73 in no process [-1]
<7> [582.174325] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.174383] ------------[ cut here ]------------
<4> [582.174384] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.174386] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.174457] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.174522] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.174527] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.174530] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.174531] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.174533] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.174537] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.174609] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.174610] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.174613] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.174614] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.174616] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.174617] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.174618] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.174619] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.174621] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.174622] CR2: 00007bda80284058 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.174624] PKRU: 55555554
<4> [582.174625] Call Trace:
<4> [582.174626] <TASK>
<4> [582.174630] ? lock_sync+0x100/0x100
<4> [582.174635] ? lock_release+0xd0/0x2b0
<4> [582.174640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.174646] process_one_work+0x239/0x760
<4> [582.174652] worker_thread+0x200/0x3f0
<4> [582.174654] ? __pfx_worker_thread+0x10/0x10
<4> [582.174657] kthread+0x10d/0x150
<4> [582.174660] ? __pfx_kthread+0x10/0x10
<4> [582.174663] ret_from_fork+0x3d4/0x480
<4> [582.174665] ? __pfx_kthread+0x10/0x10
<4> [582.174669] ret_from_fork_asm+0x1a/0x30
<4> [582.174676] </TASK>
<4> [582.174677] irq event stamp: 125009
<4> [582.174678] hardirqs last enabled at (125015): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.174681] hardirqs last disabled at (125020): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.174683] softirqs last enabled at (123990): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.174686] softirqs last disabled at (123983): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.174688] ---[ end trace 0000000000000000 ]---
<6> [582.174690] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.174761] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.174767] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.175787] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.175894] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.176001] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.176210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.176312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.176409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.176500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.176592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.176681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.176769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.176862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.176948] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.177044] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.177163] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.178169] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.188892] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.189136] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.190206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.190286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.190364] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.190438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.190509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.190579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.190648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.190718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.190787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.190856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.190925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.190994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.191062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.191132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.191200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.191274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.191350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.191426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.191502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.191586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.191668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.191748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.191830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.191908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.191988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.192066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.192145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.192226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.192315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.192398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.192482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.192565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.192652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.192737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.192822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.192904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.192984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.193062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.193142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.193224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.193316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.193398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.193476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.193553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.193635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.193712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.193790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.193866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.193944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.194019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.194096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.194175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.194253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.194341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.194417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.194496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.194572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.194652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.194736] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.194850] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.194855] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51522, lrc_seqno=51522, guc_id=0, flags=0x73 in no process [-1]
<7> [582.194857] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.194916] ------------[ cut here ]------------
<4> [582.194918] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.194919] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.194991] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.195058] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.195063] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.195066] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.195067] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.195069] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.195074] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.195143] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.195145] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.195147] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.195149] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.195150] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.195151] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.195152] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.195154] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.195155] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.195157] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.195158] PKRU: 55555554
<4> [582.195159] Call Trace:
<4> [582.195161] <TASK>
<4> [582.195165] ? lock_sync+0x100/0x100
<4> [582.195169] ? lock_release+0xd0/0x2b0
<4> [582.195175] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.195180] process_one_work+0x239/0x760
<4> [582.195187] worker_thread+0x200/0x3f0
<4> [582.195189] ? __pfx_worker_thread+0x10/0x10
<4> [582.195192] kthread+0x10d/0x150
<4> [582.195195] ? __pfx_kthread+0x10/0x10
<4> [582.195199] ret_from_fork+0x3d4/0x480
<4> [582.195201] ? __pfx_kthread+0x10/0x10
<4> [582.195204] ret_from_fork_asm+0x1a/0x30
<4> [582.195211] </TASK>
<4> [582.195213] irq event stamp: 436557
<4> [582.195214] hardirqs last enabled at (436563): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.195217] hardirqs last disabled at (436568): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.195219] softirqs last enabled at (435486): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.195221] softirqs last disabled at (435479): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.195224] ---[ end trace 0000000000000000 ]---
<7> [582.197004] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.197111] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [582.197210] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51577, lrc_seqno=51577, guc_id=0, flags=0x73 in no process [-1]
<7> [582.197213] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.197278] ------------[ cut here ]------------
<4> [582.197280] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.197281] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.197355] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.197419] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.197424] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.197427] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.197428] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.197429] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.197434] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.197505] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.197507] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.197509] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.197511] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.197512] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.197513] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.197514] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.197516] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.197517] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.197519] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.197520] PKRU: 55555554
<4> [582.197521] Call Trace:
<4> [582.197522] <TASK>
<4> [582.197526] ? lock_sync+0x100/0x100
<4> [582.197530] ? lock_release+0xd0/0x2b0
<4> [582.197536] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.197542] process_one_work+0x239/0x760
<4> [582.197548] worker_thread+0x200/0x3f0
<4> [582.197551] ? __pfx_worker_thread+0x10/0x10
<4> [582.197553] kthread+0x10d/0x150
<4> [582.197556] ? __pfx_kthread+0x10/0x10
<4> [582.197560] ret_from_fork+0x3d4/0x480
<4> [582.197562] ? __pfx_kthread+0x10/0x10
<4> [582.197565] ret_from_fork_asm+0x1a/0x30
<4> [582.197572] </TASK>
<4> [582.197573] irq event stamp: 439239
<4> [582.197575] hardirqs last enabled at (439245): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.197577] hardirqs last disabled at (439250): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.197580] softirqs last enabled at (438472): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.197582] softirqs last disabled at (438465): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.197584] ---[ end trace 0000000000000000 ]---
<6> [582.197586] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.197656] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.197662] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.197709] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.197911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.198001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.198095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.198187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.198279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.198365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.198450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.198537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.198620] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.198714] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.198832] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.199840] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.210275] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.210527] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.211762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.211844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.211922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.212003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.212081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.212159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.212235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.212320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.212396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.212470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.212545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.212619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.212695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.212770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.212844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.212917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.212992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.213064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.213141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.213226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.213319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.213400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.213482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.213560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.213639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.213717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.213795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.213874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.213954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.214032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.214110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.214192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.214283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.214371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.214456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.214538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.214618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.214696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.214774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.214850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.214930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.215008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.215090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.215173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.215256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.215348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.215427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.215504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.215584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.215660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.215736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.215816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.215895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.215978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.216056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.216137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.216215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.216297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.216377] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.216490] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.216493] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51577, lrc_seqno=51577, guc_id=0, flags=0x73 in no process [-1]
<7> [582.216496] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.216554] ------------[ cut here ]------------
<4> [582.216556] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.216557] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.216628] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.216693] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.216697] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.216700] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.216701] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.216703] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.216708] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.216778] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.216780] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.216782] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.216783] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.216785] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.216786] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.216787] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.216788] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.216790] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.216791] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.216793] PKRU: 55555554
<4> [582.216794] Call Trace:
<4> [582.216795] <TASK>
<4> [582.216799] ? lock_sync+0x100/0x100
<4> [582.216803] ? lock_release+0xd0/0x2b0
<4> [582.216808] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.216814] process_one_work+0x239/0x760
<4> [582.216820] worker_thread+0x200/0x3f0
<4> [582.216823] ? __pfx_worker_thread+0x10/0x10
<4> [582.216825] kthread+0x10d/0x150
<4> [582.216828] ? __pfx_kthread+0x10/0x10
<4> [582.216832] ret_from_fork+0x3d4/0x480
<4> [582.216834] ? __pfx_kthread+0x10/0x10
<4> [582.216837] ret_from_fork_asm+0x1a/0x30
<4> [582.216844] </TASK>
<4> [582.216846] irq event stamp: 442353
<4> [582.216847] hardirqs last enabled at (442359): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.216850] hardirqs last disabled at (442364): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.216852] softirqs last enabled at (441314): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.216854] softirqs last disabled at (441307): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.216856] ---[ end trace 0000000000000000 ]---
<6> [582.216858] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.216927] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.216933] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.218184] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.218292] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.218397] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.218602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.218690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.218784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.218875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.218967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.219057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.219145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.219234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.219326] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.219422] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.219538] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.220542] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.231265] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.231530] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.232562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.232642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.232719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.232798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.232875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.232952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.233028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.233105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.233184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.233262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.233353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.233431] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.233509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.233587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.233663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.233739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.233814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.233888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.233961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.234044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.234125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.234206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.234294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.234375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.234454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.234531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.234609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.234688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.234768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.234846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.234925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.235003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.235088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.235177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.235264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.235361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.235443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.235520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.235597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.235674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.235753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.235832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.235909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.235986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.236072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.236153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.236231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.236316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.236394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.236471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.236546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.236626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.236706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.236788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.236868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.236947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.237025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.237102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.237180] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.237296] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.237300] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51577, lrc_seqno=51577, guc_id=0, flags=0x73 in no process [-1]
<7> [582.237303] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.237361] ------------[ cut here ]------------
<4> [582.237362] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.237364] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.237436] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.237511] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.237517] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.237520] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.237522] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.237523] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.237529] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.237606] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.237608] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.237610] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.237611] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.237613] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.237614] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.237615] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.237617] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.237618] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.237620] CR2: 00007bda80284058 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.237621] PKRU: 55555554
<4> [582.237622] Call Trace:
<4> [582.237624] <TASK>
<4> [582.237628] ? lock_sync+0x100/0x100
<4> [582.237632] ? lock_release+0xd0/0x2b0
<4> [582.237638] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.237643] process_one_work+0x239/0x760
<4> [582.237650] worker_thread+0x200/0x3f0
<4> [582.237652] ? __pfx_worker_thread+0x10/0x10
<4> [582.237655] kthread+0x10d/0x150
<4> [582.237658] ? __pfx_kthread+0x10/0x10
<4> [582.237661] ret_from_fork+0x3d4/0x480
<4> [582.237663] ? __pfx_kthread+0x10/0x10
<4> [582.237667] ret_from_fork_asm+0x1a/0x30
<4> [582.237674] </TASK>
<4> [582.237675] irq event stamp: 128427
<4> [582.237676] hardirqs last enabled at (128433): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.237679] hardirqs last disabled at (128438): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.237681] softirqs last enabled at (127416): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.237684] softirqs last disabled at (127411): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.237686] ---[ end trace 0000000000000000 ]---
<5> [582.238772] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51632, lrc_seqno=51632, guc_id=0, flags=0x73 in no process [-1]
<7> [582.238777] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.238879] ------------[ cut here ]------------
<4> [582.238881] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.238882] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.238955] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.239030] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.239037] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.239040] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.239041] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.239042] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.239047] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.239117] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.239119] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.239121] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.239122] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.239124] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.239125] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.239126] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.239127] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.239130] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.239131] CR2: 00007c3624005ff8 CR3: 0000000124a85001 CR4: 0000000000f72ef0
<4> [582.239133] PKRU: 55555554
<4> [582.239135] Call Trace:
<4> [582.239137] <TASK>
<4> [582.239142] ? lock_sync+0x100/0x100
<4> [582.239146] ? lock_release+0xd0/0x2b0
<4> [582.239151] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.239157] process_one_work+0x239/0x760
<4> [582.239163] worker_thread+0x200/0x3f0
<4> [582.239166] ? __pfx_worker_thread+0x10/0x10
<4> [582.239168] kthread+0x10d/0x150
<4> [582.239171] ? __pfx_kthread+0x10/0x10
<4> [582.239175] ret_from_fork+0x3d4/0x480
<4> [582.239177] ? __pfx_kthread+0x10/0x10
<4> [582.239180] ret_from_fork_asm+0x1a/0x30
<4> [582.239187] </TASK>
<4> [582.239188] irq event stamp: 444909
<4> [582.239189] hardirqs last enabled at (444915): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.239192] hardirqs last disabled at (444920): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.239194] softirqs last enabled at (442472): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [582.239197] softirqs last disabled at (442470): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [582.239199] ---[ end trace 0000000000000000 ]---
<7> [582.239153] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<6> [582.239201] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.239277] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.239286] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.239259] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.239454] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.239668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.239764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.239859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.239948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.240035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.240119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.240206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.240301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.240388] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.240484] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.240602] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.241622] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.252276] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.252528] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.253561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.253644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.253722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.253799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.253876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.253952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.254027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.254102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.254178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.254257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.254369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.254446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.254522] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.254598] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.254672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.254747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.254820] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.254894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.254968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.255051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.255133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.255214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.255302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.255385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.255466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.255546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.255624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.255704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.255783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.255863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.255942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.256021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.256107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.256197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.256287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.256370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.256451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.256530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.256608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.256684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.256764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.256846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.256929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.257008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.257089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.257167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.257244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.257329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.257409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.257490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.257569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.257649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.257728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.257806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.257883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.257964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.258043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.258121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.258202] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.258320] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.258325] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51632, lrc_seqno=51632, guc_id=0, flags=0x73 in no process [-1]
<7> [582.258327] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.258386] ------------[ cut here ]------------
<4> [582.258387] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.258388] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:41/12316
<4> [582.258459] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.258524] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.258529] CPU: 8 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.258532] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.258533] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.258535] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.258540] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.258608] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.258610] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.258612] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.258614] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.258615] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.258616] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.258617] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.258619] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.258620] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.258622] CR2: 00007ca99c097138 CR3: 0000000124a85003 CR4: 0000000000f72ef0
<4> [582.258623] PKRU: 55555554
<4> [582.258625] Call Trace:
<4> [582.258626] <TASK>
<4> [582.258630] ? lock_sync+0x100/0x100
<4> [582.258634] ? lock_release+0xd0/0x2b0
<4> [582.258640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.258646] process_one_work+0x239/0x760
<4> [582.258652] worker_thread+0x200/0x3f0
<4> [582.258655] ? __pfx_worker_thread+0x10/0x10
<4> [582.258657] kthread+0x10d/0x150
<4> [582.258660] ? __pfx_kthread+0x10/0x10
<4> [582.258664] ret_from_fork+0x3d4/0x480
<4> [582.258666] ? __pfx_kthread+0x10/0x10
<4> [582.258669] ret_from_fork_asm+0x1a/0x30
<4> [582.258677] </TASK>
<4> [582.258678] irq event stamp: 165661
<4> [582.258679] hardirqs last enabled at (165667): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.258682] hardirqs last disabled at (165672): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.258684] softirqs last enabled at (164552): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.258687] softirqs last disabled at (164545): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.258689] ---[ end trace 0000000000000000 ]---
<6> [582.258691] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.258761] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.258767] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.260234] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.260357] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.260464] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.260668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.260756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.260850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.260938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.261028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.261115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.261202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.261294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.261376] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.261472] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.261589] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.262595] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.273611] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.273854] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.274907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.274980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.275050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.275121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.275190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.275259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.275347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.275422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.275496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.275572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.275651] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.275728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.275804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.275880] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.275955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.276030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.276103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.276177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.276251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.276345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.276428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.276508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.276590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.276671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.276754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.276834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.276913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.276994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.277075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.277156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.277238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.277329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.277415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.277499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.277583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.277665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.277745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.277823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.277903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.277984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.278066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.278145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.278224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.278309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.278390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.278466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.278543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.278624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.278705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.278783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.278859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.278938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.279015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.279093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.279169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.279246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.279330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.279406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.279484] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.279595] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.279600] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51632, lrc_seqno=51632, guc_id=0, flags=0x73 in no process [-1]
<7> [582.279602] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.279660] ------------[ cut here ]------------
<4> [582.279661] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.279663] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.279734] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.279799] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.279803] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.279806] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.279807] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.279809] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.279814] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.279883] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.279885] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.279887] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.279888] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.279890] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.279891] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.279892] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.279893] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.279895] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.279896] CR2: 00007bda80284058 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.279898] PKRU: 55555554
<4> [582.279899] Call Trace:
<4> [582.279900] <TASK>
<4> [582.279904] ? lock_sync+0x100/0x100
<4> [582.279909] ? lock_release+0xd0/0x2b0
<4> [582.279914] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.279919] process_one_work+0x239/0x760
<4> [582.279926] worker_thread+0x200/0x3f0
<4> [582.279928] ? __pfx_worker_thread+0x10/0x10
<4> [582.279931] kthread+0x10d/0x150
<4> [582.279934] ? __pfx_kthread+0x10/0x10
<4> [582.279937] ret_from_fork+0x3d4/0x480
<4> [582.279939] ? __pfx_kthread+0x10/0x10
<4> [582.279943] ret_from_fork_asm+0x1a/0x30
<4> [582.279950] </TASK>
<4> [582.279951] irq event stamp: 132189
<4> [582.279952] hardirqs last enabled at (132195): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.279955] hardirqs last disabled at (132200): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.279957] softirqs last enabled at (131328): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.279960] softirqs last disabled at (131317): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.279962] ---[ end trace 0000000000000000 ]---
<5> [582.281302] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51687, lrc_seqno=51687, guc_id=0, flags=0x73 in no process [-1]
<7> [582.281310] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.281413] ------------[ cut here ]------------
<4> [582.281414] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.281416] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [582.281505] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling
<7> [582.281473] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [582.281548] x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.281628] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.281632] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.281634] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.281636] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [582.281622] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [582.281643] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.281760] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.281762] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.281765] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.281767] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.281774] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.281776] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.281778] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.281779] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [582.281781] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.281782] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.281783] PKRU: 55555554
<4> [582.281785] Call Trace:
<4> [582.281786] <TASK>
<4> [582.281790] ? lock_sync+0x100/0x100
<4> [582.281794] ? lock_release+0xd0/0x2b0
<4> [582.281800] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.281806] process_one_work+0x239/0x760
<4> [582.281812] worker_thread+0x200/0x3f0
<4> [582.281814] ? __pfx_worker_thread+0x10/0x10
<4> [582.281817] kthread+0x10d/0x150
<4> [582.281820] ? __pfx_kthread+0x10/0x10
<4> [582.281823] ret_from_fork+0x3d4/0x480
<4> [582.281825] ? __pfx_kthread+0x10/0x10
<4> [582.281829] ret_from_fork_asm+0x1a/0x30
<4> [582.281836] </TASK>
<4> [582.281837] irq event stamp: 134959
<4> [582.281838] hardirqs last enabled at (134965): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.281841] hardirqs last disabled at (134970): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.281843] softirqs last enabled at (131328): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.281846] softirqs last disabled at (131317): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.281848] ---[ end trace 0000000000000000 ]---
<6> [582.281850] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.281924] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.281931] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.282036] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.282243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.282344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.282436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.282528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.282617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.282704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.282791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.282880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.282963] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.283059] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.283177] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.284193] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.294915] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.295166] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.296210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.296298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.296376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.296455] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.296532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.296608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.296682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.296757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.296832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.296907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.296981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.297055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.297128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.297201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.297280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.297354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.297428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.297502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.297578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.297661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.297745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.297827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.297909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.297988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.298066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.298145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.298222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.298316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.298401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.298483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.298565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.298646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.298732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.298817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.298903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.298984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.299064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.299142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.299220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.299308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.299393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.299475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.299555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.299634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.299716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.299795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.299872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.299949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.300027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.300104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.300180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.300260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.300350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.300429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.300506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.300586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.300662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.300740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.300820] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.300934] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.300938] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51687, lrc_seqno=51687, guc_id=0, flags=0x73 in no process [-1]
<7> [582.300940] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.300999] ------------[ cut here ]------------
<4> [582.301000] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.301001] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [582.301073] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.301137] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.301142] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.301144] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.301146] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.301147] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.301152] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.301221] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.301223] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.301225] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.301227] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.301228] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.301229] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.301230] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.301232] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [582.301233] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.301235] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.301236] PKRU: 55555554
<4> [582.301237] Call Trace:
<4> [582.301238] <TASK>
<4> [582.301242] ? lock_sync+0x100/0x100
<4> [582.301247] ? lock_release+0xd0/0x2b0
<4> [582.301252] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.301258] process_one_work+0x239/0x760
<4> [582.301264] worker_thread+0x200/0x3f0
<4> [582.301266] ? __pfx_worker_thread+0x10/0x10
<4> [582.301272] kthread+0x10d/0x150
<4> [582.301275] ? __pfx_kthread+0x10/0x10
<4> [582.301279] ret_from_fork+0x3d4/0x480
<4> [582.301281] ? __pfx_kthread+0x10/0x10
<4> [582.301285] ret_from_fork_asm+0x1a/0x30
<4> [582.301292] </TASK>
<4> [582.301294] irq event stamp: 138071
<4> [582.301295] hardirqs last enabled at (138077): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.301298] hardirqs last disabled at (138082): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.301300] softirqs last enabled at (137092): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.301303] softirqs last disabled at (137085): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.301305] ---[ end trace 0000000000000000 ]---
<6> [582.301307] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.301376] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.301383] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.302621] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.302729] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.302837] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.303063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.303156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.303251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.303353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.303440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.303526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.303611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.303698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.303785] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.303880] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.303998] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.305006] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.315280] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.315528] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.316609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.316692] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.316771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.316849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.316926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.317004] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.317082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.317157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.317232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.317319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.317394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.317467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.317541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.317614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.317688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.317761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.317835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.317951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.318070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.318190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.318321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.318436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.318557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.318674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.318774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.318856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.318938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.319022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.319104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.319186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.319266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.319371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.319455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.319545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.319635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.319722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.319805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.319885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.319964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.320042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.320123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.320203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.320287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.320368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.320453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.320532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.320611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.320693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.320775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.320854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.320932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.321015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.321097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.321177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.321255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.321347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.321424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.321501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.321582] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.321696] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.321701] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51687, lrc_seqno=51687, guc_id=0, flags=0x73 in no process [-1]
<7> [582.321703] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.321757] ------------[ cut here ]------------
<4> [582.321759] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.321760] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:41/12316
<4> [582.321824] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.321884] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.321889] CPU: 9 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.321891] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.321893] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.321894] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.321899] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.321962] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.321964] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.321966] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.321967] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.321969] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.321970] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.321971] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.321972] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [582.321974] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.321975] CR2: 00007a6c03de62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.321976] PKRU: 55555554
<4> [582.321977] Call Trace:
<4> [582.321978] <TASK>
<4> [582.321982] ? lock_sync+0x100/0x100
<4> [582.321987] ? lock_release+0xd0/0x2b0
<4> [582.321992] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.321997] process_one_work+0x239/0x760
<4> [582.322003] worker_thread+0x200/0x3f0
<4> [582.322005] ? __pfx_worker_thread+0x10/0x10
<4> [582.322007] kthread+0x10d/0x150
<4> [582.322010] ? __pfx_kthread+0x10/0x10
<4> [582.322013] ret_from_fork+0x3d4/0x480
<4> [582.322015] ? __pfx_kthread+0x10/0x10
<4> [582.322018] ret_from_fork_asm+0x1a/0x30
<4> [582.322025] </TASK>
<4> [582.322026] irq event stamp: 173779
<4> [582.322027] hardirqs last enabled at (173785): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.322029] hardirqs last disabled at (173790): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.322032] softirqs last enabled at (172824): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.322034] softirqs last disabled at (172817): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.322036] ---[ end trace 0000000000000000 ]---
<5> [582.323298] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51742, lrc_seqno=51742, guc_id=0, flags=0x73 in no process [-1]
<7> [582.323310] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.323657] ------------[ cut here ]------------
<4> [582.323659] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.323661] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:41/12316
<4> [582.323743] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.323845] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.323853] CPU: 8 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.323857] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.323859] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.323861] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.323870] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.323949] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.323950] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.323953] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.323954] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.323955] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.323956] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.323958] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.323959] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.323960] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.323962] CR2: 00007ca99c097138 CR3: 000000011ea2d001 CR4: 0000000000f72ef0
<4> [582.323963] PKRU: 55555554
<4> [582.323964] Call Trace:
<4> [582.323966] <TASK>
<4> [582.323969] ? lock_sync+0x100/0x100
<4> [582.323974] ? lock_release+0xd0/0x2b0
<4> [582.323979] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.323985] process_one_work+0x239/0x760
<4> [582.323991] worker_thread+0x200/0x3f0
<4> [582.323993] ? __pfx_worker_thread+0x10/0x10
<4> [582.323996] kthread+0x10d/0x150
<4> [582.323998] ? __pfx_kthread+0x10/0x10
<4> [582.324002] ret_from_fork+0x3d4/0x480
<4> [582.324004] ? __pfx_kthread+0x10/0x10
<4> [582.324007] ret_from_fork_asm+0x1a/0x30
<4> [582.324014] </TASK>
<4> [582.324016] irq event stamp: 176717
<4> [582.324017] hardirqs last enabled at (176723): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.324019] hardirqs last disabled at (176728): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.324021] softirqs last enabled at (172824): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.324024] softirqs last disabled at (172817): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.324026] ---[ end trace 0000000000000000 ]---
<6> [582.324028] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [582.324055] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<6> [582.324140] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.324149] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.324314] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.324436] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.324644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.324736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.324831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.324922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.325010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.325098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.325189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.325289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.325375] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.325471] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.325588] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.326598] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.337280] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.337535] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.338611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.338685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.338756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.338827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.338897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.338966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.339036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.339108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.339179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.339249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.339337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.339413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.339489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.339563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.339641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.339720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.339797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.339872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.339948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.340030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.340112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.340194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.340281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.340360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.340437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.340514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.340591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.340670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.340749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.340827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.340906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.340985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.341070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.341159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.341246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.341342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.341424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.341503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.341582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.341660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.341741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.341819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.341898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.341975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.342062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.342144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.342223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.342325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.342404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.342481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.342557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.342637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.342714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.342791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.342867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.342949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.343028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.343107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.343188] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.343306] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.343312] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51742, lrc_seqno=51742, guc_id=0, flags=0x73 in no process [-1]
<7> [582.343314] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.343373] ------------[ cut here ]------------
<4> [582.343374] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.343375] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:41/12316
<4> [582.343447] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.343512] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.343517] CPU: 8 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.343520] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.343521] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.343522] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.343527] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.343598] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.343600] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.343602] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.343603] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.343605] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.343606] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.343607] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.343608] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.343610] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.343611] CR2: 00007ca99c097138 CR3: 000000011ea2d001 CR4: 0000000000f72ef0
<4> [582.343613] PKRU: 55555554
<4> [582.343614] Call Trace:
<4> [582.343615] <TASK>
<4> [582.343619] ? lock_sync+0x100/0x100
<4> [582.343624] ? lock_release+0xd0/0x2b0
<4> [582.343629] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.343635] process_one_work+0x239/0x760
<4> [582.343641] worker_thread+0x200/0x3f0
<4> [582.343644] ? __pfx_worker_thread+0x10/0x10
<4> [582.343646] kthread+0x10d/0x150
<4> [582.343649] ? __pfx_kthread+0x10/0x10
<4> [582.343653] ret_from_fork+0x3d4/0x480
<4> [582.343655] ? __pfx_kthread+0x10/0x10
<4> [582.343658] ret_from_fork_asm+0x1a/0x30
<4> [582.343665] </TASK>
<4> [582.343667] irq event stamp: 179983
<4> [582.343668] hardirqs last enabled at (179989): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.343671] hardirqs last disabled at (179994): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.343673] softirqs last enabled at (179168): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.343675] softirqs last disabled at (179161): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.343678] ---[ end trace 0000000000000000 ]---
<6> [582.343679] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.343750] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.343755] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.344162] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.344368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.344453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.344544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.344633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.344721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.344808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.344894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.344981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.345063] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.345160] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.345282] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.346291] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.357014] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.357258] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.358325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.358400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.358471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.358542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.358613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.358683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.358752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.358823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.358895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.358966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.359036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.359106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.359176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.359244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.359325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.359400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.359474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.359548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.359622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.359704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.359784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.359866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.359947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.360024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.360102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.360178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.360254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.360343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.360424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.360502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.360581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.360660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.360744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.360828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.360913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.360993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.361073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.361149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.361225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.361312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.361395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.361475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.361552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.361630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.361711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.361788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.361866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.361941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.362019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.362094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.362172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.362255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.362348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.362426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.362503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.362581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.362656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.362733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.362817] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.362929] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.362933] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51742, lrc_seqno=51742, guc_id=0, flags=0x73 in no process [-1]
<7> [582.362935] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.362994] ------------[ cut here ]------------
<4> [582.362995] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.362996] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:41/12316
<4> [582.363068] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.363132] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.363136] CPU: 8 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.363139] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.363140] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.363141] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.363146] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.363216] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.363217] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.363220] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.363221] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.363222] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.363224] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.363225] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.363226] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.363228] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.363229] CR2: 00007ca99c097138 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [582.363230] PKRU: 55555554
<4> [582.363232] Call Trace:
<4> [582.363233] <TASK>
<4> [582.363237] ? lock_sync+0x100/0x100
<4> [582.363241] ? lock_release+0xd0/0x2b0
<4> [582.363246] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.363252] process_one_work+0x239/0x760
<4> [582.363258] worker_thread+0x200/0x3f0
<4> [582.363260] ? __pfx_worker_thread+0x10/0x10
<4> [582.363263] kthread+0x10d/0x150
<4> [582.363266] ? __pfx_kthread+0x10/0x10
<4> [582.363273] ret_from_fork+0x3d4/0x480
<4> [582.363275] ? __pfx_kthread+0x10/0x10
<4> [582.363279] ret_from_fork_asm+0x1a/0x30
<4> [582.363286] </TASK>
<4> [582.363287] irq event stamp: 183117
<4> [582.363289] hardirqs last enabled at (183123): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.363291] hardirqs last disabled at (183128): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.363294] softirqs last enabled at (182228): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.363296] softirqs last disabled at (182217): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.363299] ---[ end trace 0000000000000000 ]---
<7> [582.364850] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.364956] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [582.365054] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51797, lrc_seqno=51797, guc_id=0, flags=0x73 in no process [-1]
<7> [582.365056] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.365115] ------------[ cut here ]------------
<4> [582.365116] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.365118] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:41/12316
<4> [582.365190] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.365254] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.365259] CPU: 8 UID: 0 PID: 12316 Comm: kworker/u64:41 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.365261] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.365263] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.365264] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.365269] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.365353] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.365355] RSP: 0018:ffffc90007be7ca0 EFLAGS: 00010246
<4> [582.365357] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.365359] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.365361] RBP: ffffc90007be7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.365362] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.365363] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.365365] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.365366] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.365368] CR2: 00007ca99c097138 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [582.365369] PKRU: 55555554
<4> [582.365370] Call Trace:
<4> [582.365371] <TASK>
<4> [582.365375] ? lock_sync+0x100/0x100
<4> [582.365379] ? lock_release+0xd0/0x2b0
<4> [582.365385] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.365390] process_one_work+0x239/0x760
<4> [582.365397] worker_thread+0x200/0x3f0
<4> [582.365399] ? __pfx_worker_thread+0x10/0x10
<4> [582.365402] kthread+0x10d/0x150
<4> [582.365404] ? __pfx_kthread+0x10/0x10
<4> [582.365408] ret_from_fork+0x3d4/0x480
<4> [582.365410] ? __pfx_kthread+0x10/0x10
<4> [582.365414] ret_from_fork_asm+0x1a/0x30
<4> [582.365421] </TASK>
<4> [582.365422] irq event stamp: 185865
<4> [582.365424] hardirqs last enabled at (185871): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.365426] hardirqs last disabled at (185876): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.365429] softirqs last enabled at (183206): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.365431] softirqs last disabled at (183199): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.365433] ---[ end trace 0000000000000000 ]---
<6> [582.365435] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.365505] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.365511] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.365749] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.365960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.366052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.366148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.366238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.366360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.366445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.366531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.366617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.366700] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.366795] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.366912] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.367920] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.378277] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.378521] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.379761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.379834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.379904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.379976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.380045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.380115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.380185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.380254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.380338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.380410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.380479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.380547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.380614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.380684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.380754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.380824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.380893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.380961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.381030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.381104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.381178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.381251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.381340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.381415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.381486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.381557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.381626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.381698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.381770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.381841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.381917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.381993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.382073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.382153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.382231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.382315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.382394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.382468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.382542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.382613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.382686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.382757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.382828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.382898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.382971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.383041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.383111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.383181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.383256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.383341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.383414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.383486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.383556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.383627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.383699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.383771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.383842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.383912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.383984] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.384088] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.384944] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51797, lrc_seqno=51797, guc_id=0, flags=0x73 in no process [-1]
<7> [582.384947] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.385004] ------------[ cut here ]------------
<4> [582.385006] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.385009] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.385079] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.385147] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.385152] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.385155] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.385156] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.385158] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.385163] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.385233] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.385235] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.385237] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.385239] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.385240] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.385241] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.385243] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.385244] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.385246] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.385247] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.385248] PKRU: 55555554
<4> [582.385250] Call Trace:
<4> [582.385251] <TASK>
<4> [582.385255] ? lock_sync+0x100/0x100
<4> [582.385260] ? lock_release+0xd0/0x2b0
<4> [582.385265] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.385275] process_one_work+0x239/0x760
<4> [582.385281] worker_thread+0x200/0x3f0
<4> [582.385284] ? __pfx_worker_thread+0x10/0x10
<4> [582.385287] kthread+0x10d/0x150
<4> [582.385290] ? __pfx_kthread+0x10/0x10
<4> [582.385294] ret_from_fork+0x3d4/0x480
<4> [582.385296] ? __pfx_kthread+0x10/0x10
<4> [582.385300] ret_from_fork_asm+0x1a/0x30
<4> [582.385308] </TASK>
<4> [582.385309] irq event stamp: 453011
<4> [582.385310] hardirqs last enabled at (453017): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.385313] hardirqs last disabled at (453022): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.385315] softirqs last enabled at (452956): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.385318] softirqs last disabled at (452951): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.385320] ---[ end trace 0000000000000000 ]---
<6> [582.385322] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.385391] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.385398] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.385413] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.385514] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.385808] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.386027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.386127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.386227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.386343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.386436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.386527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.386619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.386716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.386805] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.386908] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.387031] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.388041] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.398279] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.398530] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.399615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.399699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.399781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.399864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.399946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.400028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.400109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.400188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.400267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.400360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.400440] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.400519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.400599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.400682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.400764] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.400843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.400923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.401001] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.401081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.401168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.401255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.401352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.401441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.401526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.401609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.401692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.401778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.401866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.401954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.402039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.402123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.402208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.402302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.402391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.402480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.402566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.402649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.402732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.402815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.402896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.402980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.403063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.403143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.403224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.403322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.403409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.403497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.403582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.403668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.403750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.403831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.403916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.403998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.404080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.404160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.404242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.404329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.404411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.404500] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.404617] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.404621] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51797, lrc_seqno=51797, guc_id=0, flags=0x73 in no process [-1]
<7> [582.404624] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.404686] ------------[ cut here ]------------
<4> [582.404687] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.404688] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:54/12329
<4> [582.404765] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.404832] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.404837] CPU: 3 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.404840] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.404841] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.404843] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.404848] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.404923] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.404925] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.404927] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.404929] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.404930] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.404931] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.404933] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.404934] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [582.404936] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.404937] CR2: 0000572c990f4a88 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.404939] PKRU: 55555554
<4> [582.404940] Call Trace:
<4> [582.404941] <TASK>
<4> [582.404945] ? lock_sync+0x100/0x100
<4> [582.404950] ? lock_release+0xd0/0x2b0
<4> [582.404955] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.404961] process_one_work+0x239/0x760
<4> [582.404967] worker_thread+0x200/0x3f0
<4> [582.404970] ? __pfx_worker_thread+0x10/0x10
<4> [582.404973] kthread+0x10d/0x150
<4> [582.404976] ? __pfx_kthread+0x10/0x10
<4> [582.404979] ret_from_fork+0x3d4/0x480
<4> [582.404981] ? __pfx_kthread+0x10/0x10
<4> [582.404985] ret_from_fork_asm+0x1a/0x30
<4> [582.404992] </TASK>
<4> [582.404994] irq event stamp: 141755
<4> [582.404995] hardirqs last enabled at (141761): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.404998] hardirqs last disabled at (141766): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.405000] softirqs last enabled at (140716): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.405002] softirqs last disabled at (140709): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.405005] ---[ end trace 0000000000000000 ]---
<5> [582.406321] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51852, lrc_seqno=51852, guc_id=0, flags=0x73 in no process [-1]
<7> [582.406332] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.406455] ------------[ cut here ]------------
<4> [582.406457] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.406458] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.406541] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof
<7> [582.406524] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [582.406598] kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.406658] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.406662] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.406663] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.406668] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.406676] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<7> [582.406670] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [582.406792] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.406794] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.406798] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.406800] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.406802] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.406803] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.406805] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.406807] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.406809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.406811] CR2: 00007bda80284058 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.406813] PKRU: 55555554
<4> [582.406815] Call Trace:
<4> [582.406822] <TASK>
<4> [582.406827] ? lock_sync+0x100/0x100
<4> [582.406832] ? lock_release+0xd0/0x2b0
<4> [582.406837] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.406843] process_one_work+0x239/0x760
<4> [582.406850] worker_thread+0x200/0x3f0
<4> [582.406852] ? __pfx_worker_thread+0x10/0x10
<4> [582.406855] kthread+0x10d/0x150
<4> [582.406858] ? __pfx_kthread+0x10/0x10
<4> [582.406861] ret_from_fork+0x3d4/0x480
<4> [582.406863] ? __pfx_kthread+0x10/0x10
<4> [582.406867] ret_from_fork_asm+0x1a/0x30
<4> [582.406874] </TASK>
<4> [582.406875] irq event stamp: 144433
<4> [582.406876] hardirqs last enabled at (144439): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.406879] hardirqs last disabled at (144444): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.406881] softirqs last enabled at (140716): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.406884] softirqs last disabled at (140709): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.406886] ---[ end trace 0000000000000000 ]---
<6> [582.406888] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.406962] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.406970] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.407105] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.407318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.407410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.407502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.407590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.407677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.407764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.407849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.407936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.408019] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.408116] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.408232] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.409251] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.419282] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.419533] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.420620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.420702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.420780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.420857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.420934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.421009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.421084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.421160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.421237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.421329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.421406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.421482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.421557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.421633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.421705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.421779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.421856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.421934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.422009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.422092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.422175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.422255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.422354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.422436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.422517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.422596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.422674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.422754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.422835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.422914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.422994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.423072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.423157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.423242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.423340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.423422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.423501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.423580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.423658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.423738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.423824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.423906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.423986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.424064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.424147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.424225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.424316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.424393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.424473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.424554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.424635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.424718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.424797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.424876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.424955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.425040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.425119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.425199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.425287] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.425401] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.425405] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51852, lrc_seqno=51852, guc_id=0, flags=0x73 in no process [-1]
<7> [582.425408] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.425467] ------------[ cut here ]------------
<4> [582.425468] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.425469] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.425541] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.425606] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.425610] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.425613] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.425614] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.425616] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.425621] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.425692] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.425693] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.425695] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.425697] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.425698] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.425699] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.425701] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.425702] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.425703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.425705] CR2: 00007bda80284058 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [582.425706] PKRU: 55555554
<4> [582.425707] Call Trace:
<4> [582.425709] <TASK>
<4> [582.425712] ? lock_sync+0x100/0x100
<4> [582.425717] ? lock_release+0xd0/0x2b0
<4> [582.425722] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.425728] process_one_work+0x239/0x760
<4> [582.425734] worker_thread+0x200/0x3f0
<4> [582.425737] ? __pfx_worker_thread+0x10/0x10
<4> [582.425739] kthread+0x10d/0x150
<4> [582.425742] ? __pfx_kthread+0x10/0x10
<4> [582.425745] ret_from_fork+0x3d4/0x480
<4> [582.425747] ? __pfx_kthread+0x10/0x10
<4> [582.425751] ret_from_fork_asm+0x1a/0x30
<4> [582.425758] </TASK>
<4> [582.425759] irq event stamp: 147557
<4> [582.425760] hardirqs last enabled at (147563): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.425763] hardirqs last disabled at (147568): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.425765] softirqs last enabled at (146538): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.425768] softirqs last disabled at (146531): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.425770] ---[ end trace 0000000000000000 ]---
<6> [582.425772] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.425841] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.425847] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.427088] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.427193] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.427307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.427511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.427600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.427692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.427780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.427866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.427951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.428037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.428125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.428207] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.428306] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.428423] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.429427] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.440149] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.440392] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.441472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.441545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.441617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.441689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.441759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.441829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.441899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.441971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.442042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.442111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.442179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.442247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.442326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.442401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.442475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.442548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.442622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.442695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.442768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.442851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.442933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.443012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.443092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.443170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.443250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.443337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.443414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.443495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.443575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.443653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.443731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.443809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.443892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.443976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.444065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.444149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.444230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.444315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.444398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.444479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.444561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.444640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.444719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.444795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.444876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.444952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.445032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.445112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.445192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.445268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.445358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.445438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.445518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.445598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.445677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.445757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.445832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.445909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.445992] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.446103] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.446107] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51852, lrc_seqno=51852, guc_id=0, flags=0x73 in no process [-1]
<7> [582.446110] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.446169] ------------[ cut here ]------------
<4> [582.446170] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.446171] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.446243] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.446312] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.446317] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.446319] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.446321] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.446322] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.446327] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.446396] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.446398] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.446400] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.446402] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.446403] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.446404] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.446405] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.446407] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.446408] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.446410] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.446411] PKRU: 55555554
<4> [582.446412] Call Trace:
<4> [582.446413] <TASK>
<4> [582.446417] ? lock_sync+0x100/0x100
<4> [582.446422] ? lock_release+0xd0/0x2b0
<4> [582.446427] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.446433] process_one_work+0x239/0x760
<4> [582.446439] worker_thread+0x200/0x3f0
<4> [582.446441] ? __pfx_worker_thread+0x10/0x10
<4> [582.446444] kthread+0x10d/0x150
<4> [582.446447] ? __pfx_kthread+0x10/0x10
<4> [582.446450] ret_from_fork+0x3d4/0x480
<4> [582.446452] ? __pfx_kthread+0x10/0x10
<4> [582.446456] ret_from_fork_asm+0x1a/0x30
<4> [582.446463] </TASK>
<4> [582.446464] irq event stamp: 456473
<4> [582.446465] hardirqs last enabled at (456479): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.446468] hardirqs last disabled at (456484): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.446470] softirqs last enabled at (455382): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.446472] softirqs last disabled at (455377): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.446475] ---[ end trace 0000000000000000 ]---
<5> [582.447869] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51907, lrc_seqno=51907, guc_id=0, flags=0x73 in no process [-1]
<7> [582.447872] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.447932] ------------[ cut here ]------------
<4> [582.447933] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.447935] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.448008] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.448071] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.448076] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.448078] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.448079] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.448081] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.448085] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.448153] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.448154] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.448156] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.448158] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.448159] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.448160] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.448161] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.448163] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.448164] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.448166] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.448167] PKRU: 55555554
<4> [582.448168] Call Trace:
<4> [582.448169] <TASK>
<4> [582.448173] ? lock_sync+0x100/0x100
<4> [582.448177] ? lock_release+0xd0/0x2b0
<4> [582.448183] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.448188] process_one_work+0x239/0x760
<4> [582.448194] worker_thread+0x200/0x3f0
<4> [582.448197] ? __pfx_worker_thread+0x10/0x10
<4> [582.448199] kthread+0x10d/0x150
<4> [582.448202] ? __pfx_kthread+0x10/0x10
<4> [582.448206] ret_from_fork+0x3d4/0x480
<4> [582.448208] ? __pfx_kthread+0x10/0x10
<4> [582.448211] ret_from_fork_asm+0x1a/0x30
<4> [582.448218] </TASK>
<4> [582.448219] irq event stamp: 459135
<4> [582.448221] hardirqs last enabled at (459141): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.448223] hardirqs last disabled at (459146): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.448225] softirqs last enabled at (455382): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.448228] softirqs last disabled at (455377): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.448230] ---[ end trace 0000000000000000 ]---
<6> [582.448232] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [582.448321] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.448425] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [582.448525] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.448530] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.448577] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.448781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.448872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.448964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.449053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.449141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.449230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.449327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.449417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.449501] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.449595] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.449712] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.450719] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.461279] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.461531] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.462618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.462698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.462775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.462852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.462929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.463006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.463082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.463159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.463237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.463323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.463400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.463476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.463552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.463626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.463700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.463774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.463847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.463922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.463996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.464077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.464158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.464240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.464330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.464411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.464490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.464567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.464643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.464722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.464801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.464879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.464957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.465035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.465118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.465201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.465289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.465375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.465458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.465539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.465618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.465695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.465775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.465854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.465931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.466007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.466087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.466164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.466245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.466334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.466415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.466493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.466570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.466649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.466726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.466803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.466879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.466958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.467038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.467118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.467200] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.467316] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.467320] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51907, lrc_seqno=51907, guc_id=0, flags=0x73 in no process [-1]
<7> [582.467322] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.467381] ------------[ cut here ]------------
<4> [582.467383] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.467384] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.467456] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.467520] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.467524] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.467527] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.467528] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.467529] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.467534] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.467605] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.467607] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.467609] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.467610] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.467611] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.467613] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.467614] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.467615] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.467617] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.467618] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.467620] PKRU: 55555554
<4> [582.467621] Call Trace:
<4> [582.467622] <TASK>
<4> [582.467626] ? lock_sync+0x100/0x100
<4> [582.467630] ? lock_release+0xd0/0x2b0
<4> [582.467635] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.467641] process_one_work+0x239/0x760
<4> [582.467647] worker_thread+0x200/0x3f0
<4> [582.467650] ? __pfx_worker_thread+0x10/0x10
<4> [582.467652] kthread+0x10d/0x150
<4> [582.467655] ? __pfx_kthread+0x10/0x10
<4> [582.467658] ret_from_fork+0x3d4/0x480
<4> [582.467660] ? __pfx_kthread+0x10/0x10
<4> [582.467664] ret_from_fork_asm+0x1a/0x30
<4> [582.467671] </TASK>
<4> [582.467672] irq event stamp: 462265
<4> [582.467673] hardirqs last enabled at (462271): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.467676] hardirqs last disabled at (462276): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.467678] softirqs last enabled at (461442): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.467681] softirqs last disabled at (461435): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.467683] ---[ end trace 0000000000000000 ]---
<6> [582.467685] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.467754] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.467760] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.469015] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.469121] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.469228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.469472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.469565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.469659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.469749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.469838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.469925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.470013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.470100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.470183] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.470287] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.470412] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.471423] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.481282] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.481528] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.482543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.482626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.482704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.482782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.482858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.482932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.483007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.483082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.483156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.483230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.483318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.483404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.483482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.483558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.483634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.483710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.483785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.483859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.483932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.484014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.484097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.484177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.484258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.484353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.484441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.484518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.484598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.484682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.484765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.484845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.484926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.485006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.485091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.485175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.485260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.485362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.485453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.485532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.485611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.485688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.485769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.485847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.485925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.486002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.486082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.486159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.486235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.486322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.486415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.486495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.486573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.486653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.486731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.486809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.486886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.486964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.487042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.487118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.487198] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.487320] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.487326] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51907, lrc_seqno=51907, guc_id=0, flags=0x73 in no process [-1]
<7> [582.487328] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.487387] ------------[ cut here ]------------
<4> [582.487388] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.487390] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#1: kworker/u64:84/12359
<4> [582.487461] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.487528] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.487533] CPU: 1 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.487536] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.487537] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.487539] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.487544] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.487613] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.487615] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.487618] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.487619] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.487620] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.487622] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.487623] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.487624] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000
<4> [582.487626] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.487627] CR2: 00007c361c005e18 CR3: 0000000124a85002 CR4: 0000000000f72ef0
<4> [582.487629] PKRU: 55555554
<4> [582.487630] Call Trace:
<4> [582.487631] <TASK>
<4> [582.487635] ? lock_sync+0x100/0x100
<4> [582.487640] ? lock_release+0xd0/0x2b0
<4> [582.487645] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.487651] process_one_work+0x239/0x760
<4> [582.487657] worker_thread+0x200/0x3f0
<4> [582.487660] ? __pfx_worker_thread+0x10/0x10
<4> [582.487662] kthread+0x10d/0x150
<4> [582.487665] ? __pfx_kthread+0x10/0x10
<4> [582.487669] ret_from_fork+0x3d4/0x480
<4> [582.487671] ? __pfx_kthread+0x10/0x10
<4> [582.487674] ret_from_fork_asm+0x1a/0x30
<4> [582.487681] </TASK>
<4> [582.487682] irq event stamp: 934503
<4> [582.487684] hardirqs last enabled at (934509): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.487687] hardirqs last disabled at (934514): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.487689] softirqs last enabled at (933400): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.487691] softirqs last disabled at (933391): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.487693] ---[ end trace 0000000000000000 ]---
<5> [582.489205] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51962, lrc_seqno=51962, guc_id=0, flags=0x73 in no process [-1]
<7> [582.489208] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.489271] ------------[ cut here ]------------
<4> [582.489277] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.489280] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#1: kworker/u64:84/12359
<4> [582.489361] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.489426] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.489430] CPU: 1 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.489433] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.489434] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.489436] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.489441] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.489512] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.489514] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.489516] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.489517] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.489518] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.489520] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.489521] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.489522] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000
<4> [582.489524] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.489525] CR2: 00007c361c005e18 CR3: 0000000124a85002 CR4: 0000000000f72ef0
<4> [582.489527] PKRU: 55555554
<4> [582.489528] Call Trace:
<4> [582.489529] <TASK>
<4> [582.489533] ? lock_sync+0x100/0x100
<4> [582.489537] ? lock_release+0xd0/0x2b0
<4> [582.489542] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.489548] process_one_work+0x239/0x760
<4> [582.489554] worker_thread+0x200/0x3f0
<4> [582.489556] ? __pfx_worker_thread+0x10/0x10
<4> [582.489559] kthread+0x10d/0x150
<4> [582.489561] ? __pfx_kthread+0x10/0x10
<4> [582.489565] ret_from_fork+0x3d4/0x480
<4> [582.489567] ? __pfx_kthread+0x10/0x10
<4> [582.489570] ret_from_fork_asm+0x1a/0x30
<4> [582.489578] </TASK>
<4> [582.489579] irq event stamp: 937223
<4> [582.489580] hardirqs last enabled at (937229): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.489582] hardirqs last disabled at (937234): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.489585] softirqs last enabled at (933400): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.489587] softirqs last disabled at (933391): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.489589] ---[ end trace 0000000000000000 ]---
<6> [582.489591] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.489663] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.489669] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.489684] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.489790] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.489897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.490106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.490198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.490299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.490390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.490476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.490562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.490653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.490743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.490827] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.490925] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.491043] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.492050] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.502282] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.502540] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.503580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.503661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.503740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.503819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.503897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.503977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.504056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.504134] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.504210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.504323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.504401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.504475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.504549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.504624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.504698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.504772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.504845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.504920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.504993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.505075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.505157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.505239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.505334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.505412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.505488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.505565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.505643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.505724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.505804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.505883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.505962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.506042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.506132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.506219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.506315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.506396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.506477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.506554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.506632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.506708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.506788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.506866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.506943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.507020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.507101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.507178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.507255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.507345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.507426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.507505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.507582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.507663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.507741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.507819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.507896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.507974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.508051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.508127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.508207] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.508328] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.508332] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51962, lrc_seqno=51962, guc_id=0, flags=0x73 in no process [-1]
<7> [582.508335] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.508393] ------------[ cut here ]------------
<4> [582.508394] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.508396] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.508468] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.508533] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.508538] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.508541] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.508542] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.508544] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.508549] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.508619] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.508621] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.508623] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.508624] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.508626] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.508627] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.508628] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.508629] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.508631] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.508632] CR2: 00007bda80284058 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [582.508634] PKRU: 55555554
<4> [582.508635] Call Trace:
<4> [582.508636] <TASK>
<4> [582.508640] ? lock_sync+0x100/0x100
<4> [582.508645] ? lock_release+0xd0/0x2b0
<4> [582.508650] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.508656] process_one_work+0x239/0x760
<4> [582.508662] worker_thread+0x200/0x3f0
<4> [582.508665] ? __pfx_worker_thread+0x10/0x10
<4> [582.508667] kthread+0x10d/0x150
<4> [582.508670] ? __pfx_kthread+0x10/0x10
<4> [582.508673] ret_from_fork+0x3d4/0x480
<4> [582.508675] ? __pfx_kthread+0x10/0x10
<4> [582.508679] ret_from_fork_asm+0x1a/0x30
<4> [582.508686] </TASK>
<4> [582.508687] irq event stamp: 153161
<4> [582.508688] hardirqs last enabled at (153167): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.508691] hardirqs last disabled at (153172): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.508693] softirqs last enabled at (152052): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.508696] softirqs last disabled at (152043): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.508698] ---[ end trace 0000000000000000 ]---
<6> [582.508700] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.508770] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.508775] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.509183] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.509383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.509481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.509573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.509662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.509748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.509834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.509922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.510011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.510094] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.510189] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.510313] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.511319] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.521280] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.521528] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.522617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.522699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.522777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.522853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.522928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.523004] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.523078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.523152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.523226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.523308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.523381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.523454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.523528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.523603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.523677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.523754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.523830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.523905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.523980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.524061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.524143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.524222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.524310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.524388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.524466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.524547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.524627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.524708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.524790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.524870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.524949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.525028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.525113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.525197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.525283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.525363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.525443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.525521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.525598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.525678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.525763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.525845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.525923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.526001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.526082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.526160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.526238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.526329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.526407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.526484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.526563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.526645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.526725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.526804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.526883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.526962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.527039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.527117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.527197] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.527312] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.527316] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=51962, lrc_seqno=51962, guc_id=0, flags=0x73 in no process [-1]
<7> [582.527318] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.527378] ------------[ cut here ]------------
<4> [582.527379] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.527380] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.527453] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.527517] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.527521] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.527524] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.527525] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.527527] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.527532] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.527603] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.527605] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.527607] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.527608] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.527610] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.527611] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.527612] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.527613] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.527615] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.527616] CR2: 00007bda80284058 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [582.527618] PKRU: 55555554
<4> [582.527619] Call Trace:
<4> [582.527620] <TASK>
<4> [582.527624] ? lock_sync+0x100/0x100
<4> [582.527628] ? lock_release+0xd0/0x2b0
<4> [582.527633] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.527639] process_one_work+0x239/0x760
<4> [582.527645] worker_thread+0x200/0x3f0
<4> [582.527648] ? __pfx_worker_thread+0x10/0x10
<4> [582.527650] kthread+0x10d/0x150
<4> [582.527653] ? __pfx_kthread+0x10/0x10
<4> [582.527657] ret_from_fork+0x3d4/0x480
<4> [582.527659] ? __pfx_kthread+0x10/0x10
<4> [582.527662] ret_from_fork_asm+0x1a/0x30
<4> [582.527669] </TASK>
<4> [582.527671] irq event stamp: 156261
<4> [582.527672] hardirqs last enabled at (156267): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.527674] hardirqs last disabled at (156272): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.527677] softirqs last enabled at (155152): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.527679] softirqs last disabled at (155145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.527681] ---[ end trace 0000000000000000 ]---
<5> [582.529382] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52017, lrc_seqno=52017, guc_id=0, flags=0x73 in no process [-1]
<7> [582.529385] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.529445] ------------[ cut here ]------------
<4> [582.529446] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.529448] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.529518] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.529581] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.529586] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.529588] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.529589] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.529591] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.529595] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.529664] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.529665] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.529667] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.529669] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.529670] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.529671] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.529672] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.529674] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.529675] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.529676] CR2: 00007bda80284058 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [582.529678] PKRU: 55555554
<4> [582.529679] Call Trace:
<4> [582.529680] <TASK>
<4> [582.529684] ? lock_sync+0x100/0x100
<4> [582.529688] ? lock_release+0xd0/0x2b0
<4> [582.529693] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.529699] process_one_work+0x239/0x760
<4> [582.529705] worker_thread+0x200/0x3f0
<4> [582.529708] ? __pfx_worker_thread+0x10/0x10
<4> [582.529710] kthread+0x10d/0x150
<4> [582.529713] ? __pfx_kthread+0x10/0x10
<4> [582.529716] ret_from_fork+0x3d4/0x480
<4> [582.529718] ? __pfx_kthread+0x10/0x10
<4> [582.529722] ret_from_fork_asm+0x1a/0x30
<4> [582.529729] </TASK>
<4> [582.529730] irq event stamp: 158911
<4> [582.529731] hardirqs last enabled at (158917): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.529734] hardirqs last disabled at (158922): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.529736] softirqs last enabled at (155152): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.529738] softirqs last disabled at (155145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.529740] ---[ end trace 0000000000000000 ]---
<6> [582.529742] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.529810] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.529816] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.529831] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.529932] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.530050] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.530257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.530389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.530489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.530577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.530663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.530749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.530840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.530930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.531014] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.531110] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.531227] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.532244] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.542283] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.542533] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.543599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.543679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.543757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.543835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.543912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.543988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.544065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.544145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.544223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.544324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.544405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.544481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.544554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.544631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.544707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.544781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.544854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.544928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.545002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.545084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.545165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.545245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.545344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.545434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.545513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.545590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.545666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.545746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.545827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.545906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.545989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.546071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.546157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.546243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.546340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.546429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.546511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.546589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.546667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.546749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.546832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.546913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.546992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.547070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.547150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.547228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.547317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.547403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.547480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.547556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.547633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.547712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.547788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.547865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.547942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.548022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.548102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.548180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.548265] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.548405] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.548409] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52017, lrc_seqno=52017, guc_id=0, flags=0x73 in no process [-1]
<7> [582.548412] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.548472] ------------[ cut here ]------------
<4> [582.548473] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.548474] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#1: kworker/u64:84/12359
<4> [582.548547] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.548612] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.548617] CPU: 1 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.548620] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.548621] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.548622] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.548627] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.548697] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.548698] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.548701] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.548702] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.548703] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.548705] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.548706] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.548707] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000
<4> [582.548709] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.548710] CR2: 00007c361c005e18 CR3: 0000000124a85002 CR4: 0000000000f72ef0
<4> [582.548712] PKRU: 55555554
<4> [582.548713] Call Trace:
<4> [582.548714] <TASK>
<4> [582.548718] ? lock_sync+0x100/0x100
<4> [582.548722] ? lock_release+0xd0/0x2b0
<4> [582.548728] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.548733] process_one_work+0x239/0x760
<4> [582.548740] worker_thread+0x200/0x3f0
<4> [582.548743] ? __pfx_worker_thread+0x10/0x10
<4> [582.548745] kthread+0x10d/0x150
<4> [582.548748] ? __pfx_kthread+0x10/0x10
<4> [582.548752] ret_from_fork+0x3d4/0x480
<4> [582.548754] ? __pfx_kthread+0x10/0x10
<4> [582.548757] ret_from_fork_asm+0x1a/0x30
<4> [582.548764] </TASK>
<4> [582.548765] irq event stamp: 942757
<4> [582.548766] hardirqs last enabled at (942763): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.548769] hardirqs last disabled at (942768): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.548772] softirqs last enabled at (941648): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.548774] softirqs last disabled at (941639): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.548776] ---[ end trace 0000000000000000 ]---
<6> [582.548778] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.548848] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.548854] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.549256] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.549483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.549571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.549662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.549750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.549837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.549922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.550008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.550100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.550184] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.550287] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.550412] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.551422] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.561284] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.561532] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.562581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.562661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.562739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.562817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.562893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.562969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.563046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.563121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.563195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.563270] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.563367] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.563449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.563526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.563604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.563680] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.563756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.563831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.563905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.563980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.564063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.564144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.564223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.564317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.564406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.564488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.564569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.564649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.564731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.564812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.564889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.564971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.565055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.565143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.565229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.565326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.565417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.565499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.565577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.565658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.565737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.565819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.565899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.565977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.566053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.566133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.566210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.566295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.566385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.566467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.566545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.566621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.566701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.566779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.566856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.566932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.567010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.567085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.567161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.567240] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.567370] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.567375] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52017, lrc_seqno=52017, guc_id=0, flags=0x73 in no process [-1]
<7> [582.567377] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.567435] ------------[ cut here ]------------
<4> [582.567437] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.567438] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#1: kworker/u64:84/12359
<4> [582.567510] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.567574] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.567579] CPU: 1 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.567582] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.567583] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.567584] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.567589] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.567659] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.567661] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.567663] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.567664] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.567666] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.567667] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.567668] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.567669] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000
<4> [582.567671] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.567672] CR2: 00007c361c005e18 CR3: 0000000124a85002 CR4: 0000000000f72ef0
<4> [582.567674] PKRU: 55555554
<4> [582.567675] Call Trace:
<4> [582.567676] <TASK>
<4> [582.567680] ? lock_sync+0x100/0x100
<4> [582.567684] ? lock_release+0xd0/0x2b0
<4> [582.567690] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.567695] process_one_work+0x239/0x760
<4> [582.567701] worker_thread+0x200/0x3f0
<4> [582.567704] ? __pfx_worker_thread+0x10/0x10
<4> [582.567706] kthread+0x10d/0x150
<4> [582.567709] ? __pfx_kthread+0x10/0x10
<4> [582.567713] ret_from_fork+0x3d4/0x480
<4> [582.567715] ? __pfx_kthread+0x10/0x10
<4> [582.567718] ret_from_fork_asm+0x1a/0x30
<4> [582.567726] </TASK>
<4> [582.567727] irq event stamp: 945881
<4> [582.567728] hardirqs last enabled at (945887): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.567731] hardirqs last disabled at (945892): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.567733] softirqs last enabled at (944772): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.567735] softirqs last disabled at (944763): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.567737] ---[ end trace 0000000000000000 ]---
<5> [582.569222] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52072, lrc_seqno=52072, guc_id=0, flags=0x73 in no process [-1]
<7> [582.569225] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.569303] ------------[ cut here ]------------
<4> [582.569305] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.569306] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.569386] Modules linked in: xe
<7> [582.569336] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [582.569389] snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [582.569454] [last unloaded: snd_hda_intel]
<4> [582.569457] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.569460] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.569461] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.569463] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [582.569441] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [582.569468] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.569541] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.569542] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.569544] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.569546] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.569547] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.569548] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.569550] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.569551] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.569553] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.569554] CR2: 00007c3624005ff8 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.569555] PKRU: 55555554
<4> [582.569557] Call Trace:
<4> [582.569558] <TASK>
<4> [582.569562] ? lock_sync+0x100/0x100
<4> [582.569566] ? lock_release+0xd0/0x2b0
<4> [582.569571] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.569577] process_one_work+0x239/0x760
<4> [582.569583] worker_thread+0x200/0x3f0
<4> [582.569586] ? __pfx_worker_thread+0x10/0x10
<4> [582.569588] kthread+0x10d/0x150
<4> [582.569591] ? __pfx_kthread+0x10/0x10
<4> [582.569595] ret_from_fork+0x3d4/0x480
<4> [582.569597] ? __pfx_kthread+0x10/0x10
<4> [582.569600] ret_from_fork_asm+0x1a/0x30
<4> [582.569608] </TASK>
<4> [582.569609] irq event stamp: 466273
<4> [582.569610] hardirqs last enabled at (466279): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.569613] hardirqs last disabled at (466284): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.569615] softirqs last enabled at (465498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.569617] softirqs last disabled at (465493): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.569619] ---[ end trace 0000000000000000 ]---
<6> [582.569621] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.569694] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.569700] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.569897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.570107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.570200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.570297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.570386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.570474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.570562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.570651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.570740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.570824] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.570919] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.571034] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.572046] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.582282] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.582534] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.583577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.583657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.583734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.583812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.583889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.583966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.584042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.584116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.584190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.584264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.584353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.584427] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.584502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.584577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.584652] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.584726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.584800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.584876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.584953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.585036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.585118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.585198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.585282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.585361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.585438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.585519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.585600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.585681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.585762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.585842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.585921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.586001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.586085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.586170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.586256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.586350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.586431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.586510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.586587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.586667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.586751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.586832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.586910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.586987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.587068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.587145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.587223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.587309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.587391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.587468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.587545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.587625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.587703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.587780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.587857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.587934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.588009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.588089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.588170] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.589129] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.589133] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52072, lrc_seqno=52072, guc_id=0, flags=0x73 in no process [-1]
<7> [582.589136] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.589198] ------------[ cut here ]------------
<4> [582.589199] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.589201] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:84/12359
<4> [582.589273] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.589344] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.589349] CPU: 8 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.589352] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.589353] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.589355] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.589360] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.589430] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.589432] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.589434] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.589436] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.589437] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.589438] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.589439] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.589441] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.589442] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.589444] CR2: 00007ca99c097138 CR3: 000000011ea2d001 CR4: 0000000000f72ef0
<4> [582.589445] PKRU: 55555554
<4> [582.589446] Call Trace:
<4> [582.589447] <TASK>
<4> [582.589451] ? lock_sync+0x100/0x100
<4> [582.589456] ? lock_release+0xd0/0x2b0
<4> [582.589462] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.589467] process_one_work+0x239/0x760
<4> [582.589473] worker_thread+0x200/0x3f0
<4> [582.589476] ? __pfx_worker_thread+0x10/0x10
<4> [582.589479] kthread+0x10d/0x150
<4> [582.589482] ? __pfx_kthread+0x10/0x10
<4> [582.589485] ret_from_fork+0x3d4/0x480
<4> [582.589487] ? __pfx_kthread+0x10/0x10
<4> [582.589491] ret_from_fork_asm+0x1a/0x30
<4> [582.589498] </TASK>
<4> [582.589499] irq event stamp: 950775
<4> [582.589500] hardirqs last enabled at (950781): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.589503] hardirqs last disabled at (950786): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.589506] softirqs last enabled at (949846): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.589508] softirqs last disabled at (949839): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.589511] ---[ end trace 0000000000000000 ]---
<6> [582.589513] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.589583] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.589589] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.590131] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.590231] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.590343] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.590772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.590865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.590959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.591046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.591132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.591216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.591307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.591393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.591476] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.591575] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.591694] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.592702] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.603282] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.603533] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.604563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.604642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.604720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.604797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.604874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.604951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.605028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.605104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.605181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.605257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.605343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.605417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.605491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.605566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.605640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.605714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.605787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.605861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.605935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.606016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.606100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.606181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.606262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.606353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.606430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.606507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.606583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.606663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.606742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.606820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.606899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.606977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.607061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.607146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.607233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.607324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.607406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.607484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.607562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.607640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.607720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.607799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.607877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.607953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.608034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.608111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.608189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.608265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.608355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.608432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.608512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.608594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.608673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.608751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.608827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.608905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.608981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.609057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.609139] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.609250] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.609255] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52072, lrc_seqno=52072, guc_id=0, flags=0x73 in no process [-1]
<7> [582.609257] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.610179] ------------[ cut here ]------------
<4> [582.610180] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.610182] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:84/12359
<4> [582.610254] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.610336] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.610341] CPU: 8 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.610345] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.610346] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.610347] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.610352] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.610422] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.610423] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.610426] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.610427] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.610428] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.610429] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.610431] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.610432] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.610434] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.610435] CR2: 00007ca99c097138 CR3: 000000011ea2d001 CR4: 0000000000f72ef0
<4> [582.610437] PKRU: 55555554
<4> [582.610438] Call Trace:
<4> [582.610439] <TASK>
<4> [582.610443] ? lock_sync+0x100/0x100
<4> [582.610447] ? lock_release+0xd0/0x2b0
<4> [582.610453] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.610458] process_one_work+0x239/0x760
<4> [582.610465] worker_thread+0x200/0x3f0
<4> [582.610467] ? __pfx_worker_thread+0x10/0x10
<4> [582.610470] kthread+0x10d/0x150
<4> [582.610473] ? __pfx_kthread+0x10/0x10
<4> [582.610477] ret_from_fork+0x3d4/0x480
<4> [582.610479] ? __pfx_kthread+0x10/0x10
<4> [582.610482] ret_from_fork_asm+0x1a/0x30
<4> [582.610489] </TASK>
<4> [582.610490] irq event stamp: 953911
<4> [582.610492] hardirqs last enabled at (953917): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.610495] hardirqs last disabled at (953922): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.610497] softirqs last enabled at (952896): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.610499] softirqs last disabled at (952889): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.610502] ---[ end trace 0000000000000000 ]---
<7> [582.610515] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.610618] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [582.611324] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52127, lrc_seqno=52127, guc_id=0, flags=0x73 in no process [-1]
<7> [582.611331] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.611439] ------------[ cut here ]------------
<4> [582.611441] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.611443] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:31/12306
<4> [582.611562] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.611656] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.611662] CPU: 12 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.611666] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.611668] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.611670] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.611678] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.611793] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.611796] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.611799] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.611801] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.611803] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.611805] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.611807] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.611809] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [582.611811] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.611814] CR2: 00007c3614005ff8 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [582.611816] PKRU: 55555554
<4> [582.611817] Call Trace:
<4> [582.611819] <TASK>
<4> [582.611826] ? lock_sync+0x100/0x100
<4> [582.611833] ? lock_release+0xd0/0x2b0
<4> [582.611842] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.611851] process_one_work+0x239/0x760
<4> [582.611861] worker_thread+0x200/0x3f0
<4> [582.611866] ? __pfx_worker_thread+0x10/0x10
<4> [582.611869] kthread+0x10d/0x150
<4> [582.611873] ? __pfx_kthread+0x10/0x10
<4> [582.611878] ret_from_fork+0x3d4/0x480
<4> [582.611881] ? __pfx_kthread+0x10/0x10
<4> [582.611886] ret_from_fork_asm+0x1a/0x30
<4> [582.611898] </TASK>
<4> [582.611900] irq event stamp: 468953
<4> [582.611902] hardirqs last enabled at (468959): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.611906] hardirqs last disabled at (468964): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.611909] softirqs last enabled at (468070): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.611912] softirqs last disabled at (468065): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.611916] ---[ end trace 0000000000000000 ]---
<6> [582.611918] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.612033] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.612042] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.612258] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.612547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.612637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.612729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.612817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.612904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.612989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.613074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.613160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.613242] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.613346] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.613463] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.614471] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.624284] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.624538] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.625621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.625704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.625782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.625860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.625937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.626014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.626090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.626165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.626240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.626324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.626398] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.626472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.626546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.626619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.626694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.626767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.626841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.626916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.626989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.627071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.627152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.627233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.627327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.627408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.627489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.627568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.627646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.627727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.627808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.627891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.627974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.628057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.628144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.628228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.628320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.628403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.628484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.628563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.628641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.628718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.628798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.628881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.628962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.629040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.629122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.629201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.629282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.629359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.629439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.629515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.629592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.629672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.629748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.629826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.629904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.629987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.630067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.630145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.630226] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.630345] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.630349] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52127, lrc_seqno=52127, guc_id=0, flags=0x73 in no process [-1]
<7> [582.630352] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.630410] ------------[ cut here ]------------
<4> [582.630411] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.630413] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:31/12306
<4> [582.630484] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.630548] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.630553] CPU: 10 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.630556] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.630557] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.630559] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.630564] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.630634] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.630635] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.630638] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.630639] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.630640] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.630642] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.630643] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.630644] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [582.630646] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.630647] CR2: 00007c3624005ff8 CR3: 000000011ea2d002 CR4: 0000000000f72ef0
<4> [582.630649] PKRU: 55555554
<4> [582.630650] Call Trace:
<4> [582.630651] <TASK>
<4> [582.630655] ? lock_sync+0x100/0x100
<4> [582.630659] ? lock_release+0xd0/0x2b0
<4> [582.630665] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.630671] process_one_work+0x239/0x760
<4> [582.630677] worker_thread+0x200/0x3f0
<4> [582.630680] ? __pfx_worker_thread+0x10/0x10
<4> [582.630682] kthread+0x10d/0x150
<4> [582.630685] ? __pfx_kthread+0x10/0x10
<4> [582.630689] ret_from_fork+0x3d4/0x480
<4> [582.630691] ? __pfx_kthread+0x10/0x10
<4> [582.630694] ret_from_fork_asm+0x1a/0x30
<4> [582.630701] </TASK>
<4> [582.630702] irq event stamp: 472137
<4> [582.630703] hardirqs last enabled at (472143): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.630706] hardirqs last disabled at (472148): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.630709] softirqs last enabled at (471028): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.630711] softirqs last disabled at (471019): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.630713] ---[ end trace 0000000000000000 ]---
<6> [582.630715] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.630785] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.630791] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.632036] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.632141] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.632247] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.632467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.632558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.632651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.632742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.632833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.632922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.633010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.633097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.633179] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.633274] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.633404] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.634411] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.644284] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.644533] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.645584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.645663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.645740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.645818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.645895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.645971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.646047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.646120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.646196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.646291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.646369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.646444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.646520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.646594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.646669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.646743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.646818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.646891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.646965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.647045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.647128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.647209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.647295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.647373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.647451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.647528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.647604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.647683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.647763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.647842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.647923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.648004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.648090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.648174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.648258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.648348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.648427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.648503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.648580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.648656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.648734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.648815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.648897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.648975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.649056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.649134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.649211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.649294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.649372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.649446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.649521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.649599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.649676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.649753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.649834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.649915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.649992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.650068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.650148] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.650494] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.650499] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52127, lrc_seqno=52127, guc_id=0, flags=0x73 in no process [-1]
<7> [582.650501] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.650562] ------------[ cut here ]------------
<4> [582.650563] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.650564] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.650638] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.650703] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.650708] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.650711] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.650712] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.650714] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.650719] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.650789] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.650791] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.650793] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.650795] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.650796] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.650797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.650798] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.650800] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.650801] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.650803] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [582.650804] PKRU: 55555554
<4> [582.650806] Call Trace:
<4> [582.650807] <TASK>
<4> [582.650810] ? lock_sync+0x100/0x100
<4> [582.650815] ? lock_release+0xd0/0x2b0
<4> [582.650820] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.650826] process_one_work+0x239/0x760
<4> [582.650832] worker_thread+0x200/0x3f0
<4> [582.650835] ? __pfx_worker_thread+0x10/0x10
<4> [582.650837] kthread+0x10d/0x150
<4> [582.650840] ? __pfx_kthread+0x10/0x10
<4> [582.650844] ret_from_fork+0x3d4/0x480
<4> [582.650846] ? __pfx_kthread+0x10/0x10
<4> [582.650849] ret_from_fork_asm+0x1a/0x30
<4> [582.650857] </TASK>
<4> [582.650858] irq event stamp: 162379
<4> [582.650859] hardirqs last enabled at (162385): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.650862] hardirqs last disabled at (162390): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.650864] softirqs last enabled at (161552): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.650866] softirqs last disabled at (161545): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.650869] ---[ end trace 0000000000000000 ]---
<5> [582.652252] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52182, lrc_seqno=52182, guc_id=0, flags=0x73 in no process [-1]
<7> [582.652255] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [582.652342] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.652445] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [582.652544] ------------[ cut here ]------------
<4> [582.652545] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.652546] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.652618] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.652682] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.652686] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.652689] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.652690] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.652691] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.652696] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.652765] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.652767] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.652769] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.652770] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.652772] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.652773] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.652774] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.652775] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.652777] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.652778] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [582.652780] PKRU: 55555554
<4> [582.652781] Call Trace:
<4> [582.652782] <TASK>
<4> [582.652786] ? lock_sync+0x100/0x100
<4> [582.652790] ? lock_release+0xd0/0x2b0
<4> [582.652795] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.652801] process_one_work+0x239/0x760
<4> [582.652807] worker_thread+0x200/0x3f0
<4> [582.652810] ? __pfx_worker_thread+0x10/0x10
<4> [582.652812] kthread+0x10d/0x150
<4> [582.652815] ? __pfx_kthread+0x10/0x10
<4> [582.652818] ret_from_fork+0x3d4/0x480
<4> [582.652820] ? __pfx_kthread+0x10/0x10
<4> [582.652824] ret_from_fork_asm+0x1a/0x30
<4> [582.652831] </TASK>
<4> [582.652832] irq event stamp: 164463
<4> [582.652833] hardirqs last enabled at (164469): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.652836] hardirqs last disabled at (164474): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.652838] softirqs last enabled at (161552): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.652840] softirqs last disabled at (161545): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.652843] ---[ end trace 0000000000000000 ]---
<6> [582.652844] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.652914] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.652920] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.652968] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.653171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.653262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.653368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.653458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.653547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.653638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.653728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.653818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.653903] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.653999] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.654116] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.655121] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.665283] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.665534] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.666646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.666727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.666804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.666881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.666958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.667033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.667107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.667182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.667257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.667341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.667416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.667491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.667568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.667645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.667720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.667795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.667869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.667943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.668017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.668099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.668180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.668259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.668350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.668428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.668505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.668583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.668663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.668745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.668827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.668907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.668986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.669065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.669149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.669233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.669325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.669406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.669485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.669564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.669645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.669724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.669805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.669884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.669962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.670039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.670120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.670197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.670274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.670363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.670439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.670514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.670590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.670668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.670744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.670821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.670896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.670973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.671047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.671122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.671200] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.671315] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.671319] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52182, lrc_seqno=52182, guc_id=0, flags=0x73 in no process [-1]
<7> [582.671322] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.671381] ------------[ cut here ]------------
<4> [582.671382] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.671383] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:31/12306
<4> [582.671455] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.671520] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.671525] CPU: 8 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.671528] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.671529] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.671530] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.671535] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.671606] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.671608] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.671610] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.671611] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.671613] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.671614] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.671615] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.671616] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [582.671618] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.671619] CR2: 00007ca99c097138 CR3: 0000000124a85003 CR4: 0000000000f72ef0
<4> [582.671621] PKRU: 55555554
<4> [582.671622] Call Trace:
<4> [582.671623] <TASK>
<4> [582.671627] ? lock_sync+0x100/0x100
<4> [582.671632] ? lock_release+0xd0/0x2b0
<4> [582.671637] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.671643] process_one_work+0x239/0x760
<4> [582.671649] worker_thread+0x200/0x3f0
<4> [582.671651] ? __pfx_worker_thread+0x10/0x10
<4> [582.671654] kthread+0x10d/0x150
<4> [582.671657] ? __pfx_kthread+0x10/0x10
<4> [582.671660] ret_from_fork+0x3d4/0x480
<4> [582.671662] ? __pfx_kthread+0x10/0x10
<4> [582.671666] ret_from_fork_asm+0x1a/0x30
<4> [582.671673] </TASK>
<4> [582.671674] irq event stamp: 477651
<4> [582.671675] hardirqs last enabled at (477657): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.671678] hardirqs last disabled at (477662): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.671680] softirqs last enabled at (476542): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.671683] softirqs last disabled at (476537): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.671685] ---[ end trace 0000000000000000 ]---
<6> [582.671687] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.671756] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.671763] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.673006] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.673109] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.673216] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.673433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.673525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.673619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.673708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.673800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.673890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.673977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.674066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.674151] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.674245] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.674374] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.675384] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.685285] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.685532] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.686582] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.686671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.686754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.686834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.686912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.686988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.687062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.687136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.687210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.687287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.687361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.687436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.687509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.687583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.687656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.687730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.687804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.687877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.687952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.688033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.688116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.688198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.688284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.688362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.688440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.688517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.688594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.688674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.688754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.688832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.688911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.688989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.689074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.689157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.689242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.689332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.689412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.689495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.689576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.689655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.689737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.689816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.689894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.689970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.690051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.690129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.690206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.690291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.690376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.690455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.690533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.690615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.690693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.690772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.690849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.690928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.691004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.691081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.691160] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.691274] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.692128] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52182, lrc_seqno=52182, guc_id=0, flags=0x73 in no process [-1]
<7> [582.692130] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.692191] ------------[ cut here ]------------
<4> [582.692192] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.692194] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.692265] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.692337] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.692341] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.692344] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.692346] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.692347] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.692352] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.692424] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.692425] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.692427] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.692429] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.692430] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.692431] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.692433] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.692434] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.692436] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.692437] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.692439] PKRU: 55555554
<4> [582.692440] Call Trace:
<4> [582.692441] <TASK>
<4> [582.692445] ? lock_sync+0x100/0x100
<4> [582.692449] ? lock_release+0xd0/0x2b0
<4> [582.692455] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.692460] process_one_work+0x239/0x760
<4> [582.692467] worker_thread+0x200/0x3f0
<4> [582.692470] ? __pfx_worker_thread+0x10/0x10
<4> [582.692472] kthread+0x10d/0x150
<4> [582.692475] ? __pfx_kthread+0x10/0x10
<4> [582.692479] ret_from_fork+0x3d4/0x480
<4> [582.692481] ? __pfx_kthread+0x10/0x10
<4> [582.692484] ret_from_fork_asm+0x1a/0x30
<4> [582.692491] </TASK>
<4> [582.692493] irq event stamp: 959635
<4> [582.692494] hardirqs last enabled at (959641): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.692497] hardirqs last disabled at (959646): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.692499] softirqs last enabled at (958966): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.692502] softirqs last disabled at (958961): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.692504] ---[ end trace 0000000000000000 ]---
<7> [582.692518] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.692623] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [582.693335] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52237, lrc_seqno=52237, guc_id=0, flags=0x73 in no process [-1]
<7> [582.693338] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.693397] ------------[ cut here ]------------
<4> [582.693398] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.693400] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.693471] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.693535] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.693539] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.693542] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.693543] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.693544] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.693549] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.693618] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.693620] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.693622] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.693623] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.693625] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.693626] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.693627] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.693628] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.693630] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.693631] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.693633] PKRU: 55555554
<4> [582.693634] Call Trace:
<4> [582.693635] <TASK>
<4> [582.693638] ? lock_sync+0x100/0x100
<4> [582.693643] ? lock_release+0xd0/0x2b0
<4> [582.693648] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.693653] process_one_work+0x239/0x760
<4> [582.693659] worker_thread+0x200/0x3f0
<4> [582.693662] ? __pfx_worker_thread+0x10/0x10
<4> [582.693665] kthread+0x10d/0x150
<4> [582.693667] ? __pfx_kthread+0x10/0x10
<4> [582.693671] ret_from_fork+0x3d4/0x480
<4> [582.693673] ? __pfx_kthread+0x10/0x10
<4> [582.693676] ret_from_fork_asm+0x1a/0x30
<4> [582.693683] </TASK>
<4> [582.693684] irq event stamp: 962361
<4> [582.693685] hardirqs last enabled at (962367): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.693688] hardirqs last disabled at (962372): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.693690] softirqs last enabled at (961388): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.693693] softirqs last disabled at (961377): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.693695] ---[ end trace 0000000000000000 ]---
<6> [582.693696] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.693769] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.693775] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.693924] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.694132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.694224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.694325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.694414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.694502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.694588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.694676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.694765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.694850] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.694944] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.695063] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.696074] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.706284] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.706535] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.707626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.707705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.707782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.707861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.707941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.708020] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.708095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.708171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.708246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.708334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.708412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.708488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.708564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.708640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.708716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.708790] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.708864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.708938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.709012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.709092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.709174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.709255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.709344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.709423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.709501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.709578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.709655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.709734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.709814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.709893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.709971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.710049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.710132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.710215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.710302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.710385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.710469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.710550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.710629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.710708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.710789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.710868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.710946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.711022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.711102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.711182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.711263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.711354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.711434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.711511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.711588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.711667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.711743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.711820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.711896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.711979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.712058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.712136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.712217] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.712333] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.712337] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52237, lrc_seqno=52237, guc_id=0, flags=0x73 in no process [-1]
<7> [582.712339] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.712398] ------------[ cut here ]------------
<4> [582.712400] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.712401] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:54/12329
<4> [582.712473] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.712538] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.712543] CPU: 2 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.712545] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.712547] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.712548] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.712553] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.712622] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.712624] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.712626] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.712628] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.712629] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.712630] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.712631] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.712633] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [582.712634] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.712636] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [582.712637] PKRU: 55555554
<4> [582.712638] Call Trace:
<4> [582.712640] <TASK>
<4> [582.712643] ? lock_sync+0x100/0x100
<4> [582.712648] ? lock_release+0xd0/0x2b0
<4> [582.712653] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.712659] process_one_work+0x239/0x760
<4> [582.712665] worker_thread+0x200/0x3f0
<4> [582.712668] ? __pfx_worker_thread+0x10/0x10
<4> [582.712670] kthread+0x10d/0x150
<4> [582.712673] ? __pfx_kthread+0x10/0x10
<4> [582.712676] ret_from_fork+0x3d4/0x480
<4> [582.712678] ? __pfx_kthread+0x10/0x10
<4> [582.712682] ret_from_fork_asm+0x1a/0x30
<4> [582.712689] </TASK>
<4> [582.712690] irq event stamp: 169855
<4> [582.712692] hardirqs last enabled at (169861): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.712694] hardirqs last disabled at (169866): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.712697] softirqs last enabled at (169014): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.712699] softirqs last disabled at (169007): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.712701] ---[ end trace 0000000000000000 ]---
<6> [582.712703] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.712773] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.712779] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.713185] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.713439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.713548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.713650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.713744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.713836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.713927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.714018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.714109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.714196] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.714303] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.714426] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.715444] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.726227] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.726527] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.727611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.727699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.727781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.727863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.727945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.728026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.728106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.728185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.728265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.728359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.728438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.728520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.728602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.728684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.728764] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.728844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.728924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.729002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.729080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.729168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.729254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.729346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.729432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.729519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.729605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.729688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.729769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.729854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.729939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.730022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.730105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.730189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.730281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.730371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.730460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.730547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.730634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.730720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.730806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.730890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.730975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.731058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.731140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.731221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.731313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.731400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.731485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.731567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.731652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.731732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.731813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.731898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.731980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.732062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.732147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.732235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.732325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.732408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.732493] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.732610] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.732614] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52237, lrc_seqno=52237, guc_id=0, flags=0x73 in no process [-1]
<7> [582.732617] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.732680] ------------[ cut here ]------------
<4> [582.732681] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.732682] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.732758] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.732824] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.732828] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.732832] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.732833] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.732834] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.732839] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.732914] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.732916] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.732918] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.732920] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.732922] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.732923] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.732925] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.732926] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.732928] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.732929] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.732931] PKRU: 55555554
<4> [582.732933] Call Trace:
<4> [582.732934] <TASK>
<4> [582.732938] ? lock_sync+0x100/0x100
<4> [582.732943] ? lock_release+0xd0/0x2b0
<4> [582.732949] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.732955] process_one_work+0x239/0x760
<4> [582.732961] worker_thread+0x200/0x3f0
<4> [582.732964] ? __pfx_worker_thread+0x10/0x10
<4> [582.732967] kthread+0x10d/0x150
<4> [582.732970] ? __pfx_kthread+0x10/0x10
<4> [582.732974] ret_from_fork+0x3d4/0x480
<4> [582.732976] ? __pfx_kthread+0x10/0x10
<4> [582.732980] ret_from_fork_asm+0x1a/0x30
<4> [582.732987] </TASK>
<4> [582.732989] irq event stamp: 172971
<4> [582.732990] hardirqs last enabled at (172977): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.732993] hardirqs last disabled at (172982): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.732996] softirqs last enabled at (171856): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.732998] softirqs last disabled at (171849): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.733001] ---[ end trace 0000000000000000 ]---
<5> [582.734214] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52292, lrc_seqno=52292, guc_id=0, flags=0x73 in no process [-1]
<7> [582.734220] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.734376] ------------[ cut here ]------------
<4> [582.734378] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.734379] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.734458] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.734524] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.734528] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.734531] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.734532] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.734534] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.734538] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.734609] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.734610] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.734613] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.734614] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.734615] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.734616] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.734618] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.734619] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.734620] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.734622] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.734623] PKRU: 55555554
<4> [582.734624] Call Trace:
<4> [582.734626] <TASK>
<7> [582.734570] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [582.734630] ? lock_sync+0x100/0x100
<4> [582.734634] ? lock_release+0xd0/0x2b0
<4> [582.734639] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.734644] process_one_work+0x239/0x760
<4> [582.734650] worker_thread+0x200/0x3f0
<4> [582.734653] ? __pfx_worker_thread+0x10/0x10
<4> [582.734655] kthread+0x10d/0x150
<4> [582.734658] ? __pfx_kthread+0x10/0x10
<4> [582.734661] ret_from_fork+0x3d4/0x480
<4> [582.734663] ? __pfx_kthread+0x10/0x10
<4> [582.734666] ret_from_fork_asm+0x1a/0x30
<4> [582.734673] </TASK>
<4> [582.734674] irq event stamp: 966027
<4> [582.734675] hardirqs last enabled at (966033): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.734677] hardirqs last disabled at (966038): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.734679] softirqs last enabled at (965260): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.734682] softirqs last disabled at (965209): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.734683] ---[ end trace 0000000000000000 ]---
<6> [582.734685] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [582.734689] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [582.734756] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.734762] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.734860] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.735062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.735152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.735246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.735345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.735434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.735522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.735610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.735699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.735782] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.735877] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.735994] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.737002] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.747285] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.747535] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.748609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.748690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.748767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.748845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.748923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.748999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.749077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.749155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.749230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.749313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.749387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.749462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.749537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.749611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.749685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.749760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.749833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.749906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.749979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.750062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.750144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.750223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.750337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.750419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.750497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.750575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.750651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.750731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.750811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.750890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.750972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.751054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.751143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.751228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.751320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.751404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.751484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.751561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.751638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.751714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.751795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.751877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.751957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.752036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.752119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.752198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.752282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.752361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.752441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.752521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.752599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.752680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.752759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.752836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.752912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.752989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.753064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.753140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.753219] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.753331] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.753335] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52292, lrc_seqno=52292, guc_id=0, flags=0x73 in no process [-1]
<7> [582.753338] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.753397] ------------[ cut here ]------------
<4> [582.753398] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.753399] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.753472] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.753536] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.753541] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.753543] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.753545] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.753546] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.753551] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.753620] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.753622] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.753624] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.753626] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.753627] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.753628] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.753629] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.753631] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.753632] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.753634] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.753635] PKRU: 55555554
<4> [582.753636] Call Trace:
<4> [582.753637] <TASK>
<4> [582.753641] ? lock_sync+0x100/0x100
<4> [582.753646] ? lock_release+0xd0/0x2b0
<4> [582.753651] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.753657] process_one_work+0x239/0x760
<4> [582.753663] worker_thread+0x200/0x3f0
<4> [582.753666] ? __pfx_worker_thread+0x10/0x10
<4> [582.753668] kthread+0x10d/0x150
<4> [582.753671] ? __pfx_kthread+0x10/0x10
<4> [582.753674] ret_from_fork+0x3d4/0x480
<4> [582.753676] ? __pfx_kthread+0x10/0x10
<4> [582.753680] ret_from_fork_asm+0x1a/0x30
<4> [582.753687] </TASK>
<4> [582.753688] irq event stamp: 969287
<4> [582.753689] hardirqs last enabled at (969293): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.753692] hardirqs last disabled at (969298): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.753694] softirqs last enabled at (968268): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.753697] softirqs last disabled at (968261): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.753699] ---[ end trace 0000000000000000 ]---
<6> [582.753701] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.753770] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.753777] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.755018] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.755122] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.755228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.755451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.755546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.755647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.755743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.755835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.755926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.756017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.756110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.756197] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.756302] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.756426] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.757436] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.768162] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.768411] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.769488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.769574] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.769655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.769736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.769816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.769897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.769979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.770063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.770145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.770226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.770313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.770393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.770472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.770551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.770630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.770708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.770786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.770864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.770943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.771032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.771121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.771204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.771301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.771384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.771467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.771548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.771629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.771714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.771804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.771890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.771977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.772063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.772154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.772244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.772342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.772427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.772511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.772594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.772677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.772759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.772844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.772926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.773009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.773091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.773176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.773258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.773364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.773450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.773536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.773619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.773701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.773786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.773869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.773951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.774032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.774119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.774203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.774288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.774374] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.774489] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.774493] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52292, lrc_seqno=52292, guc_id=0, flags=0x73 in no process [-1]
<7> [582.774495] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.774557] ------------[ cut here ]------------
<4> [582.774558] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.774560] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.774635] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.774700] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.774704] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.774707] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.774708] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.774710] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.774715] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.774789] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.774791] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.774793] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.774795] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.774796] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.774797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.774798] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.774800] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.774801] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.774803] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.774804] PKRU: 55555554
<4> [582.774805] Call Trace:
<4> [582.774807] <TASK>
<4> [582.774810] ? lock_sync+0x100/0x100
<4> [582.774815] ? lock_release+0xd0/0x2b0
<4> [582.774821] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.774826] process_one_work+0x239/0x760
<4> [582.774833] worker_thread+0x200/0x3f0
<4> [582.774835] ? __pfx_worker_thread+0x10/0x10
<4> [582.774838] kthread+0x10d/0x150
<4> [582.774841] ? __pfx_kthread+0x10/0x10
<4> [582.774844] ret_from_fork+0x3d4/0x480
<4> [582.774846] ? __pfx_kthread+0x10/0x10
<4> [582.774850] ret_from_fork_asm+0x1a/0x30
<4> [582.774857] </TASK>
<4> [582.774858] irq event stamp: 177829
<4> [582.774860] hardirqs last enabled at (177835): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.774863] hardirqs last disabled at (177840): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.774865] softirqs last enabled at (176806): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.774868] softirqs last disabled at (176799): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.774870] ---[ end trace 0000000000000000 ]---
<5> [582.776121] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52347, lrc_seqno=52347, guc_id=0, flags=0x73 in no process [-1]
<7> [582.776124] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.776185] ------------[ cut here ]------------
<4> [582.776186] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.776188] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.776256] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.776324] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.776328] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.776331] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.776332] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.776333] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.776338] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.776407] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.776409] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.776411] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.776412] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.776413] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.776414] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.776415] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.776417] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.776418] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.776419] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.776421] PKRU: 55555554
<4> [582.776422] Call Trace:
<4> [582.776423] <TASK>
<4> [582.776427] ? lock_sync+0x100/0x100
<4> [582.776431] ? lock_release+0xd0/0x2b0
<4> [582.776436] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.776442] process_one_work+0x239/0x760
<4> [582.776448] worker_thread+0x200/0x3f0
<4> [582.776451] ? __pfx_worker_thread+0x10/0x10
<4> [582.776453] kthread+0x10d/0x150
<4> [582.776456] ? __pfx_kthread+0x10/0x10
<4> [582.776459] ret_from_fork+0x3d4/0x480
<4> [582.776461] ? __pfx_kthread+0x10/0x10
<4> [582.776465] ret_from_fork_asm+0x1a/0x30
<4> [582.776472] </TASK>
<4> [582.776473] irq event stamp: 179921
<4> [582.776474] hardirqs last enabled at (179927): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.776477] hardirqs last disabled at (179932): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.776479] softirqs last enabled at (176806): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.776481] softirqs last disabled at (176799): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.776484] ---[ end trace 0000000000000000 ]---
<7> [582.776497] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.776602] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [582.776701] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.776773] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.776779] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.776826] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.777030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.777117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.777209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.777301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.777389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.777476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.777564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.777653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.777740] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.777838] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.777955] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.778961] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.789286] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.789536] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.790626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.790706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.790783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.790860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.790937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.791012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.791086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.791159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.791233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.791314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.791387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.791461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.791537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.791614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.791690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.791764] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.791839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.791912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.791986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.792069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.792152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.792232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.792319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.792397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.792477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.792554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.792632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.792711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.792790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.792867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.792947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.793026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.793110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.793193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.793276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.793370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.793449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.793525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.793602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.793678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.793757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.793835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.793915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.793994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.794076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.794155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.794233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.794317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.794397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.794473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.794549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.794628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.794708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.794789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.794868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.794947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.795025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.795106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.795189] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.795305] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.795309] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52347, lrc_seqno=52347, guc_id=0, flags=0x73 in no process [-1]
<7> [582.795311] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.795369] ------------[ cut here ]------------
<4> [582.795370] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.795372] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.795442] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.795507] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.795511] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.795513] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.795515] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.795516] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.795521] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.795591] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.795592] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.795595] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.795596] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.795597] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.795598] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.795600] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.795601] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.795602] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.795604] CR2: 00007ddcc54a8008 CR3: 000000011ea25002 CR4: 0000000000f72ef0
<4> [582.795605] PKRU: 55555554
<4> [582.795606] Call Trace:
<4> [582.795607] <TASK>
<4> [582.795611] ? lock_sync+0x100/0x100
<4> [582.795616] ? lock_release+0xd0/0x2b0
<4> [582.795621] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.795626] process_one_work+0x239/0x760
<4> [582.795632] worker_thread+0x200/0x3f0
<4> [582.795635] ? __pfx_worker_thread+0x10/0x10
<4> [582.795638] kthread+0x10d/0x150
<4> [582.795640] ? __pfx_kthread+0x10/0x10
<4> [582.795644] ret_from_fork+0x3d4/0x480
<4> [582.795646] ? __pfx_kthread+0x10/0x10
<4> [582.795649] ret_from_fork_asm+0x1a/0x30
<4> [582.795656] </TASK>
<4> [582.795658] irq event stamp: 183027
<4> [582.795659] hardirqs last enabled at (183033): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.795661] hardirqs last disabled at (183038): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.795664] softirqs last enabled at (181816): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.795666] softirqs last disabled at (181811): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.795668] ---[ end trace 0000000000000000 ]---
<6> [582.795670] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.795740] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.795746] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.796156] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.796362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.796448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.796538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.796626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.796714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.796800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.796887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.796974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.797055] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.797150] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.797270] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.798296] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.809017] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.809267] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.810297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.810376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.810453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.810530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.810606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.810682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.810758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.810833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.810908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.810982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.811059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.811136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.811212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.811291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.811365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.811438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.811511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.811585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.811657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.811739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.811820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.811901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.811985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.812066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.812143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.812220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.812298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.812379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.812460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.812538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.812616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.812695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.812779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.812863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.812952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.813036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.813119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.813197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.813275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.813364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.813444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.813522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.813598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.813674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.813754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.813830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.813907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.813983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.814061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.814136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.814214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.814300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.814381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.814460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.814537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.814615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.814692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.814768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.814852] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.814963] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.814967] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52347, lrc_seqno=52347, guc_id=0, flags=0x73 in no process [-1]
<7> [582.814969] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.815027] ------------[ cut here ]------------
<4> [582.815028] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.815029] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.815102] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.815166] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.815170] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.815173] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.815174] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.815175] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.815180] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.815250] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.815251] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.815253] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.815255] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.815256] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.815257] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.815259] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.815260] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.815261] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.815263] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.815264] PKRU: 55555554
<4> [582.815265] Call Trace:
<4> [582.815267] <TASK>
<4> [582.815270] ? lock_sync+0x100/0x100
<4> [582.815275] ? lock_release+0xd0/0x2b0
<4> [582.815284] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.815289] process_one_work+0x239/0x760
<4> [582.815295] worker_thread+0x200/0x3f0
<4> [582.815298] ? __pfx_worker_thread+0x10/0x10
<4> [582.815301] kthread+0x10d/0x150
<4> [582.815304] ? __pfx_kthread+0x10/0x10
<4> [582.815307] ret_from_fork+0x3d4/0x480
<4> [582.815309] ? __pfx_kthread+0x10/0x10
<4> [582.815313] ret_from_fork_asm+0x1a/0x30
<4> [582.815320] </TASK>
<4> [582.815321] irq event stamp: 186155
<4> [582.815322] hardirqs last enabled at (186161): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.815325] hardirqs last disabled at (186166): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.815327] softirqs last enabled at (184880): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.815330] softirqs last disabled at (184871): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.815332] ---[ end trace 0000000000000000 ]---
<7> [582.817007] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.817113] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [582.817212] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52402, lrc_seqno=52402, guc_id=0, flags=0x73 in no process [-1]
<7> [582.817215] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.817274] ------------[ cut here ]------------
<4> [582.817275] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.817276] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.817363] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.817427] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.817431] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.817434] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.817435] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.817436] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.817441] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.817510] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.817512] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.817514] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.817515] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.817516] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.817517] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.817518] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.817520] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.817521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.817523] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.817524] PKRU: 55555554
<4> [582.817525] Call Trace:
<4> [582.817526] <TASK>
<4> [582.817530] ? lock_sync+0x100/0x100
<4> [582.817534] ? lock_release+0xd0/0x2b0
<4> [582.817539] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.817545] process_one_work+0x239/0x760
<4> [582.817551] worker_thread+0x200/0x3f0
<4> [582.817554] ? __pfx_worker_thread+0x10/0x10
<4> [582.817556] kthread+0x10d/0x150
<4> [582.817559] ? __pfx_kthread+0x10/0x10
<4> [582.817563] ret_from_fork+0x3d4/0x480
<4> [582.817565] ? __pfx_kthread+0x10/0x10
<4> [582.817568] ret_from_fork_asm+0x1a/0x30
<4> [582.817575] </TASK>
<4> [582.817576] irq event stamp: 189501
<4> [582.817578] hardirqs last enabled at (189507): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.817580] hardirqs last disabled at (189512): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.817582] softirqs last enabled at (188752): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.817585] softirqs last disabled at (188747): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.817587] ---[ end trace 0000000000000000 ]---
<6> [582.817588] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.817659] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.817666] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.817775] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.817978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.818068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.818162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.818252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.818350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.818441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.818529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.818617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.818701] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.818798] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.818915] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.819922] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.830286] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.830536] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.831608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.831688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.831765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.831842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.831919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.831995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.832073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.832151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.832228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.832310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.832385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.832459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.832534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.832608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.832682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.832756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.832828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.832901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.832975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.833055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.833136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.833215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.833301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.833382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.833461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.833539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.833615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.833696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.833776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.833855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.833940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.834022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.834109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.834195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.834282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.834365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.834450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.834531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.834611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.834688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.834769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.834847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.834924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.835001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.835081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.835158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.835236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.835324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.835401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.835478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.835554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.835634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.835711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.835790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.835870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.835951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.836029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.836106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.836186] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.836603] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.836607] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52402, lrc_seqno=52402, guc_id=0, flags=0x73 in no process [-1]
<7> [582.836610] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.836673] ------------[ cut here ]------------
<4> [582.836674] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.836676] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.836750] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.836814] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.836819] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.836821] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.836823] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.836824] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.836829] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.836899] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.836901] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.836903] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.836905] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.836906] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.836907] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.836908] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.836910] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.836911] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.836913] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.836914] PKRU: 55555554
<4> [582.836915] Call Trace:
<4> [582.836916] <TASK>
<4> [582.836920] ? lock_sync+0x100/0x100
<4> [582.836924] ? lock_release+0xd0/0x2b0
<4> [582.836930] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.836935] process_one_work+0x239/0x760
<4> [582.836941] worker_thread+0x200/0x3f0
<4> [582.836944] ? __pfx_worker_thread+0x10/0x10
<4> [582.836947] kthread+0x10d/0x150
<4> [582.836949] ? __pfx_kthread+0x10/0x10
<4> [582.836953] ret_from_fork+0x3d4/0x480
<4> [582.836955] ? __pfx_kthread+0x10/0x10
<4> [582.836958] ret_from_fork_asm+0x1a/0x30
<4> [582.836966] </TASK>
<4> [582.836967] irq event stamp: 192629
<4> [582.836968] hardirqs last enabled at (192635): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.836971] hardirqs last disabled at (192640): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.836973] softirqs last enabled at (191704): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.836975] softirqs last disabled at (191699): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.836978] ---[ end trace 0000000000000000 ]---
<6> [582.836979] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.837049] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.837055] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.838348] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.838455] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.838563] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.839029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.839124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.839220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.839317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.839406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.839494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.839580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.839673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.839759] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.839856] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.839974] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.840979] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.851288] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.851537] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.852546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.852626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.852703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.852781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.852857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.852934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.853013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.853090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.853166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.853242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.853327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.853401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.853475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.853549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.853624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.853701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.853776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.853851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.853925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.854007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.854088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.854167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.854247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.854333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.854411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.854488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.854564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.854644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.854724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.854806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.854889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.854971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.855057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.855141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.855226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.855313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.855393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.855470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.855547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.855624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.855703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.855781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.855858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.855935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.856019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.856099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.856178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.856256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.856343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.856419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.856495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.856574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.856652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.856729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.856805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.856885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.856966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.857046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.857128] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.857238] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.857242] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52402, lrc_seqno=52402, guc_id=0, flags=0x73 in no process [-1]
<7> [582.857244] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.857315] ------------[ cut here ]------------
<4> [582.857316] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.857318] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [582.857389] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.857454] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.857459] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.857461] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.857462] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.857464] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.857469] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.857539] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.857541] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.857543] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.857544] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.857546] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.857547] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.857548] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.857549] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.857551] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.857552] CR2: 00007ddcc54a8008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.857554] PKRU: 55555554
<4> [582.857555] Call Trace:
<4> [582.857556] <TASK>
<4> [582.857560] ? lock_sync+0x100/0x100
<4> [582.857564] ? lock_release+0xd0/0x2b0
<4> [582.857570] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.857575] process_one_work+0x239/0x760
<4> [582.857581] worker_thread+0x200/0x3f0
<4> [582.857584] ? __pfx_worker_thread+0x10/0x10
<4> [582.857586] kthread+0x10d/0x150
<4> [582.857589] ? __pfx_kthread+0x10/0x10
<4> [582.857593] ret_from_fork+0x3d4/0x480
<4> [582.857595] ? __pfx_kthread+0x10/0x10
<4> [582.857598] ret_from_fork_asm+0x1a/0x30
<4> [582.857605] </TASK>
<4> [582.857607] irq event stamp: 195753
<4> [582.857608] hardirqs last enabled at (195759): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.857611] hardirqs last disabled at (195764): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.857613] softirqs last enabled at (194986): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.857615] softirqs last disabled at (194975): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.857617] ---[ end trace 0000000000000000 ]---
<5> [582.858212] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52457, lrc_seqno=52457, guc_id=0, flags=0x73 in no process [-1]
<7> [582.858384] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.858548] ------------[ cut here ]------------
<4> [582.858551] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.858554] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:31/12306
<4> [582.858673] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.858774] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.858781] CPU: 12 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.858786] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.858787] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.858790] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.858800] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.858917] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.858919] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.858923] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.858925] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.858927] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.858929] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.858931] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.858933] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [582.858935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.858937] CR2: 00007c3614005ff8 CR3: 0000000124a85002 CR4: 0000000000f72ef0
<4> [582.858940] PKRU: 55555554
<4> [582.858941] Call Trace:
<4> [582.858943] <TASK>
<4> [582.858950] ? lock_sync+0x100/0x100
<4> [582.858958] ? lock_release+0xd0/0x2b0
<4> [582.858966] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.858975] process_one_work+0x239/0x760
<4> [582.858987] worker_thread+0x200/0x3f0
<4> [582.858992] ? __pfx_worker_thread+0x10/0x10
<4> [582.858996] kthread+0x10d/0x150
<4> [582.859001] ? __pfx_kthread+0x10/0x10
<4> [582.859006] ret_from_fork+0x3d4/0x480
<4> [582.859010] ? __pfx_kthread+0x10/0x10
<4> [582.859016] ret_from_fork_asm+0x1a/0x30
<4> [582.859029] </TASK>
<4> [582.859032] irq event stamp: 486177
<4> [582.859033] hardirqs last enabled at (486183): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.859037] hardirqs last disabled at (486188): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.859040] softirqs last enabled at (485408): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.859044] softirqs last disabled at (485229): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.859047] ---[ end trace 0000000000000000 ]---
<6> [582.859050] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.859180] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.859189] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.859220] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.859330] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.859441] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.859695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.859831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.859971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.860104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.860237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.860376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.860510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.860645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.860776] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.860919] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.861086] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.862509] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.872291] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.872601] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.873847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.873968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.874087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.874209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.874337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.874458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.874579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.874699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.874819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.874940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.875060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.875181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.875306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.875427] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.875547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.875668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.875788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.875908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.876029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.876154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.876284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.876409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.876534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.876658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.876781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.876904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.877027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.877154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.877286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.877412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.877539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.877665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.877797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.877928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.878060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.878189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.878320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.878446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.878570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.878694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.878821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.878947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.879072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.879196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.879331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.879456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.879581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.879704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.879830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.879953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.880076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.880203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.880349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.880474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.880598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.880723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.880846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.880969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.881096] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.881256] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.881263] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52457, lrc_seqno=52457, guc_id=0, flags=0x73 in no process [-1]
<7> [582.881266] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.881373] ------------[ cut here ]------------
<4> [582.881375] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.881377] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:31/12306
<4> [582.881490] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.881581] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.881587] CPU: 12 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.881591] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.881593] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.881595] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.881602] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.881714] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.881717] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.881720] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.881722] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.881724] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.881726] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.881728] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.881730] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [582.881732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.881734] CR2: 00007c3614005ff8 CR3: 0000000124a85002 CR4: 0000000000f72ef0
<4> [582.881736] PKRU: 55555554
<4> [582.881738] Call Trace:
<4> [582.881739] <TASK>
<4> [582.881746] ? lock_sync+0x100/0x100
<4> [582.881753] ? lock_release+0xd0/0x2b0
<4> [582.881761] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.881770] process_one_work+0x239/0x760
<4> [582.881780] worker_thread+0x200/0x3f0
<4> [582.881784] ? __pfx_worker_thread+0x10/0x10
<4> [582.881787] kthread+0x10d/0x150
<4> [582.881791] ? __pfx_kthread+0x10/0x10
<4> [582.881796] ret_from_fork+0x3d4/0x480
<4> [582.881799] ? __pfx_kthread+0x10/0x10
<4> [582.881804] ret_from_fork_asm+0x1a/0x30
<4> [582.881816] </TASK>
<4> [582.881817] irq event stamp: 489381
<4> [582.881819] hardirqs last enabled at (489387): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.881823] hardirqs last disabled at (489392): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.881826] softirqs last enabled at (488614): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.881829] softirqs last disabled at (488607): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.881832] ---[ end trace 0000000000000000 ]---
<6> [582.881835] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.881947] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.881955] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.883586] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.883741] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.883897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.884147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.884277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.884429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.884523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.884615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.884704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.884789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.884877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.884958] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.885051] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.885168] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.886190] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.896912] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.897160] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.898252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.898344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.898424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.898502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.898580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.898654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.898728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.898803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.898878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.898952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.899026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.899100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.899175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.899249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.899345] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.899423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.899499] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.899574] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.899649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.899729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.899809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.899888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.899969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.900048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.900129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.900210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.900295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.900377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.900458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.900536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.900615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.900693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.900777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.900862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.900949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.901034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.901117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.901195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.901273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.901368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.901453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.901535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.901616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.901694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.901776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.901854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.901931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.902007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.902084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.902158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.902234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.902332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.902409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.902491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.902571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.902654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.902733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.902810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.902890] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.903238] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.903243] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52457, lrc_seqno=52457, guc_id=0, flags=0x73 in no process [-1]
<7> [582.903246] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.904276] ------------[ cut here ]------------
<4> [582.904278] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.904279] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/12306
<4> [582.904374] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.904442] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.904447] CPU: 0 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.904450] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.904451] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.904452] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.904458] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.904529] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.904531] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.904533] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.904534] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.904536] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.904537] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.904538] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.904539] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.904541] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.904543] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.904544] PKRU: 55555554
<4> [582.904545] Call Trace:
<4> [582.904546] <TASK>
<4> [582.904550] ? lock_sync+0x100/0x100
<4> [582.904555] ? lock_release+0xd0/0x2b0
<4> [582.904561] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.904566] process_one_work+0x239/0x760
<4> [582.904573] worker_thread+0x200/0x3f0
<4> [582.904576] ? __pfx_worker_thread+0x10/0x10
<4> [582.904578] kthread+0x10d/0x150
<4> [582.904581] ? __pfx_kthread+0x10/0x10
<4> [582.904585] ret_from_fork+0x3d4/0x480
<4> [582.904587] ? __pfx_kthread+0x10/0x10
<4> [582.904590] ret_from_fork_asm+0x1a/0x30
<4> [582.904598] </TASK>
<4> [582.904599] irq event stamp: 492527
<4> [582.904600] hardirqs last enabled at (492533): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.904603] hardirqs last disabled at (492538): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.904605] softirqs last enabled at (491778): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.904607] softirqs last disabled at (491771): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.904610] ---[ end trace 0000000000000000 ]---
<7> [582.904624] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.904729] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [582.905444] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52512, lrc_seqno=52512, guc_id=0, flags=0x73 in no process [-1]
<7> [582.905447] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.905507] ------------[ cut here ]------------
<4> [582.905508] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.905509] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:31/12306
<4> [582.905582] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.905647] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.905651] CPU: 0 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.905654] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.905655] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.905657] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.905662] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.905732] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.905733] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.905735] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.905737] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.905738] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.905739] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.905740] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.905742] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.905743] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.905744] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [582.905746] PKRU: 55555554
<4> [582.905747] Call Trace:
<4> [582.905748] <TASK>
<4> [582.905752] ? lock_sync+0x100/0x100
<4> [582.905756] ? lock_release+0xd0/0x2b0
<4> [582.905762] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.905767] process_one_work+0x239/0x760
<4> [582.905774] worker_thread+0x200/0x3f0
<4> [582.905777] ? __pfx_worker_thread+0x10/0x10
<4> [582.905779] kthread+0x10d/0x150
<4> [582.905782] ? __pfx_kthread+0x10/0x10
<4> [582.905786] ret_from_fork+0x3d4/0x480
<4> [582.905788] ? __pfx_kthread+0x10/0x10
<4> [582.905791] ret_from_fork_asm+0x1a/0x30
<4> [582.905799] </TASK>
<4> [582.905800] irq event stamp: 495219
<4> [582.905801] hardirqs last enabled at (495225): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.905803] hardirqs last disabled at (495230): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.905806] softirqs last enabled at (491778): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.905808] softirqs last disabled at (491771): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.905810] ---[ end trace 0000000000000000 ]---
<6> [582.905812] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.905881] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.905887] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.906101] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.906315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.906409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.906505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.906599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.906689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.906776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.906863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.906951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.907033] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.907132] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.907250] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.908266] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.918323] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.918576] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.919650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.919730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.919807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.919887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.919966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.920043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.920119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.920194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.920270] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.920362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.920437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.920511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.920586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.920659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.920733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.920808] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.920881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.920954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.921031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.921117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.921200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.921285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.921369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.921448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.921525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.921603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.921679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.921759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.921839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.921918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.921998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.922078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.922162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.922246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.922349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.922435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.922520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.922601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.922680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.922757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.922838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.922917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.922995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.923073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.923154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.923232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.923321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.923398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.923475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.923552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.923632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.923716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.923796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.923876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.923954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.924032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.924108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.924184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.924263] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.924394] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.924398] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52512, lrc_seqno=52512, guc_id=0, flags=0x73 in no process [-1]
<7> [582.924400] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.924458] ------------[ cut here ]------------
<4> [582.924459] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.924461] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.924532] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.924597] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.924601] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.924604] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.924605] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.924607] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.924612] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.924682] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.924684] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.924686] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.924687] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.924689] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.924690] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.924691] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.924693] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.924694] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.924695] CR2: 00007ca9980a5050 CR3: 0000000124a85004 CR4: 0000000000f72ef0
<4> [582.924697] PKRU: 55555554
<4> [582.924698] Call Trace:
<4> [582.924699] <TASK>
<4> [582.924703] ? lock_sync+0x100/0x100
<4> [582.924707] ? lock_release+0xd0/0x2b0
<4> [582.924713] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.924718] process_one_work+0x239/0x760
<4> [582.924724] worker_thread+0x200/0x3f0
<4> [582.924727] ? __pfx_worker_thread+0x10/0x10
<4> [582.924729] kthread+0x10d/0x150
<4> [582.924732] ? __pfx_kthread+0x10/0x10
<4> [582.924736] ret_from_fork+0x3d4/0x480
<4> [582.924738] ? __pfx_kthread+0x10/0x10
<4> [582.924741] ret_from_fork_asm+0x1a/0x30
<4> [582.924749] </TASK>
<4> [582.924750] irq event stamp: 974837
<4> [582.924751] hardirqs last enabled at (974843): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.924754] hardirqs last disabled at (974848): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.924756] softirqs last enabled at (973910): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.924758] softirqs last disabled at (973901): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.924760] ---[ end trace 0000000000000000 ]---
<6> [582.924762] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.924832] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.924839] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.925118] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.925327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.925417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.925510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.925597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.925684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.925770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.925855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.925945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.926030] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.926125] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.926241] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.927251] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.937292] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.937540] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.938778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.938857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.938934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.939011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.939090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.939170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.939247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.939337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.939414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.939489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.939563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.939637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.939711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.939785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.939858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.939930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.940004] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.940078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.940151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.940231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.940320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.940399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.940482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.940562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.940640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.940717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.940795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.940874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.940954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.941032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.941111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.941189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.941273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.941372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.941460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.941543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.941624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.941702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.941779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.941856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.941935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.942013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.942091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.942174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.942257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.942351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.942432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.942508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.942586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.942662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.942739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.942819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.942900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.942980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.943058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.943136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.943213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.943296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.943377] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.943488] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.943492] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52512, lrc_seqno=52512, guc_id=0, flags=0x73 in no process [-1]
<7> [582.943494] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.943553] ------------[ cut here ]------------
<4> [582.943554] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.943556] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.943627] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.943692] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.943696] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.943699] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.943700] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.943702] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.943706] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.943776] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.943778] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.943780] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.943782] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.943783] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.943784] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.943785] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.943787] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.943788] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.943790] CR2: 00007ca9980a5050 CR3: 0000000124a85004 CR4: 0000000000f72ef0
<4> [582.943791] PKRU: 55555554
<4> [582.943792] Call Trace:
<4> [582.943793] <TASK>
<4> [582.943797] ? lock_sync+0x100/0x100
<4> [582.943802] ? lock_release+0xd0/0x2b0
<4> [582.943807] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.943812] process_one_work+0x239/0x760
<4> [582.943818] worker_thread+0x200/0x3f0
<4> [582.943821] ? __pfx_worker_thread+0x10/0x10
<4> [582.943824] kthread+0x10d/0x150
<4> [582.943826] ? __pfx_kthread+0x10/0x10
<4> [582.943830] ret_from_fork+0x3d4/0x480
<4> [582.943832] ? __pfx_kthread+0x10/0x10
<4> [582.943836] ret_from_fork_asm+0x1a/0x30
<4> [582.943843] </TASK>
<4> [582.943844] irq event stamp: 977941
<4> [582.943845] hardirqs last enabled at (977947): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.943848] hardirqs last disabled at (977952): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.943850] softirqs last enabled at (976820): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.943852] softirqs last disabled at (976813): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.943855] ---[ end trace 0000000000000000 ]---
<5> [582.945099] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52567, lrc_seqno=52567, guc_id=0, flags=0x73 in no process [-1]
<7> [582.945102] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.945163] ------------[ cut here ]------------
<4> [582.945164] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.945166] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [582.945236] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.945309] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.945313] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.945316] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.945317] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.945318] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.945323] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.945391] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.945393] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [582.945395] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.945396] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.945397] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.945398] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.945399] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.945401] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [582.945402] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.945403] CR2: 00007ca9980a5050 CR3: 0000000124a85004 CR4: 0000000000f72ef0
<4> [582.945405] PKRU: 55555554
<4> [582.945406] Call Trace:
<4> [582.945407] <TASK>
<4> [582.945411] ? lock_sync+0x100/0x100
<4> [582.945415] ? lock_release+0xd0/0x2b0
<4> [582.945420] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.945425] process_one_work+0x239/0x760
<4> [582.945431] worker_thread+0x200/0x3f0
<4> [582.945434] ? __pfx_worker_thread+0x10/0x10
<4> [582.945436] kthread+0x10d/0x150
<4> [582.945439] ? __pfx_kthread+0x10/0x10
<4> [582.945443] ret_from_fork+0x3d4/0x480
<4> [582.945445] ? __pfx_kthread+0x10/0x10
<4> [582.945448] ret_from_fork_asm+0x1a/0x30
<4> [582.945455] </TASK>
<4> [582.945456] irq event stamp: 980095
<4> [582.945457] hardirqs last enabled at (980101): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.945460] hardirqs last disabled at (980106): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.945462] softirqs last enabled at (979294): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.945465] softirqs last disabled at (979263): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.945467] ---[ end trace 0000000000000000 ]---
<7> [582.945481] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.945586] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [582.945684] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.945758] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.945764] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.945814] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.946030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.946128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.946228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.946331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.946425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.946520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.946614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.946708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.946797] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.946898] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.947022] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.948041] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.958308] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.958574] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.959619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.959699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.959777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.959855] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.959934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.960011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.960087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.960161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.960236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.960318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.960392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.960466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.960539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.960613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.960686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.960759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.960832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.960905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.960978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.961060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.961144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.961225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.961311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.961389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.961466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.961541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.961617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.961695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.961774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.961851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.961930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.962008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.962092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.962176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.962263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.962361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.962443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.962523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.962601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.962677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.962756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.962834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.962911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.962986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.963064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.963140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.963216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.963296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.963373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.963448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.963523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.963601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.963679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.963761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.963840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.963920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.963997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.964075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.964156] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.964267] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.964271] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52567, lrc_seqno=52567, guc_id=0, flags=0x73 in no process [-1]
<7> [582.964274] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.964354] ------------[ cut here ]------------
<4> [582.964355] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.964357] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:54/12329
<4> [582.964429] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.964494] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.964499] CPU: 6 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.964502] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.964503] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.964505] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.964510] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.964580] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.964582] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.964584] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.964586] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.964587] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.964589] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.964590] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.964591] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [582.964593] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.964594] CR2: 00007ddcc7a37920 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.964596] PKRU: 55555554
<4> [582.964597] Call Trace:
<4> [582.964598] <TASK>
<4> [582.964602] ? lock_sync+0x100/0x100
<4> [582.964607] ? lock_release+0xd0/0x2b0
<4> [582.964612] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.964618] process_one_work+0x239/0x760
<4> [582.964624] worker_thread+0x200/0x3f0
<4> [582.964627] ? __pfx_worker_thread+0x10/0x10
<4> [582.964629] kthread+0x10d/0x150
<4> [582.964632] ? __pfx_kthread+0x10/0x10
<4> [582.964636] ret_from_fork+0x3d4/0x480
<4> [582.964638] ? __pfx_kthread+0x10/0x10
<4> [582.964641] ret_from_fork_asm+0x1a/0x30
<4> [582.964649] </TASK>
<4> [582.964650] irq event stamp: 204297
<4> [582.964651] hardirqs last enabled at (204303): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.964654] hardirqs last disabled at (204308): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.964656] softirqs last enabled at (203376): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.964658] softirqs last disabled at (203369): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.964661] ---[ end trace 0000000000000000 ]---
<6> [582.964663] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [582.964732] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.964738] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.966261] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [582.966384] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.966494] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.966704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.966796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.966891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.966979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.967066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.967151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.967237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.967333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.967415] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.967510] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.967627] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.968647] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [582.979626] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [582.979876] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [582.980956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [582.981035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [582.981110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [582.981186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [582.981262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [582.981348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [582.981424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [582.981502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [582.981578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [582.981654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [582.981728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [582.981803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [582.981877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [582.981950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [582.982024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [582.982096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [582.982169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [582.982242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [582.982325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [582.982407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [582.982490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [582.982572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [582.982652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [582.982730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [582.982807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [582.982887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [582.982967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [582.983048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [582.983128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [582.983207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [582.983289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [582.983368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [582.983453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [582.983537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [582.983621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [582.983701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [582.983779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [582.983856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [582.983933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [582.984009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [582.984089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [582.984168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [582.984245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [582.984331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [582.984412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [582.984489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [582.984565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [582.984641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [582.984719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [582.984794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [582.984869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [582.984947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [582.985025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [582.985101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [582.985176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [582.985255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [582.985339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [582.985414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [582.985493] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [582.985608] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [582.985613] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52567, lrc_seqno=52567, guc_id=0, flags=0x73 in no process [-1]
<7> [582.985615] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.985674] ------------[ cut here ]------------
<4> [582.985675] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.985677] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:31/12306
<4> [582.985749] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.985816] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.985821] CPU: 4 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.985824] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.985825] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.985826] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.985832] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.985901] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.985903] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [582.985905] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.985907] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.985908] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.985909] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.985911] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.985912] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [582.985914] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.985915] CR2: 00007ddcc54aa008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [582.985917] PKRU: 55555554
<4> [582.985918] Call Trace:
<4> [582.985919] <TASK>
<4> [582.985923] ? lock_sync+0x100/0x100
<4> [582.985928] ? lock_release+0xd0/0x2b0
<4> [582.985934] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.985940] process_one_work+0x239/0x760
<4> [582.985946] worker_thread+0x200/0x3f0
<4> [582.985949] ? __pfx_worker_thread+0x10/0x10
<4> [582.985951] kthread+0x10d/0x150
<4> [582.985954] ? __pfx_kthread+0x10/0x10
<4> [582.985958] ret_from_fork+0x3d4/0x480
<4> [582.985960] ? __pfx_kthread+0x10/0x10
<4> [582.985963] ret_from_fork_asm+0x1a/0x30
<4> [582.985971] </TASK>
<4> [582.985972] irq event stamp: 498745
<4> [582.985973] hardirqs last enabled at (498751): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.985976] hardirqs last disabled at (498756): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.985978] softirqs last enabled at (497796): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.985981] softirqs last disabled at (497789): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.985983] ---[ end trace 0000000000000000 ]---
<5> [582.986786] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52622, lrc_seqno=52622, guc_id=0, flags=0x73 in no process [-1]
<7> [582.986793] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [582.986908] ------------[ cut here ]------------
<4> [582.986910] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [582.986912] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [582.986988] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [582.987064] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [582.987069] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [582.987074] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [582.987075] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [582.987077] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [582.987085] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [582.987170] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [582.987172] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [582.987174] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [582.987175] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [582.987177] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [582.987178] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [582.987179] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [582.987181] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [582.987182] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [582.987184] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [582.987185] PKRU: 55555554
<4> [582.987186] Call Trace:
<4> [582.987187] <TASK>
<4> [582.987192] ? lock_sync+0x100/0x100
<4> [582.987196] ? lock_release+0xd0/0x2b0
<4> [582.987202] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [582.987208] process_one_work+0x239/0x760
<4> [582.987214] worker_thread+0x200/0x3f0
<4> [582.987217] ? __pfx_worker_thread+0x10/0x10
<4> [582.987219] kthread+0x10d/0x150
<4> [582.987222] ? __pfx_kthread+0x10/0x10
<4> [582.987226] ret_from_fork+0x3d4/0x480
<4> [582.987228] ? __pfx_kthread+0x10/0x10
<4> [582.987231] ret_from_fork_asm+0x1a/0x30
<4> [582.987239] </TASK>
<4> [582.987240] irq event stamp: 206443
<4> [582.987241] hardirqs last enabled at (206449): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [582.987244] hardirqs last disabled at (206454): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [582.987246] softirqs last enabled at (204716): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.987249] softirqs last disabled at (204707): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [582.987258] ---[ end trace 0000000000000000 ]---
<6> [582.987260] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [582.987274] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<6> [582.987347] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [582.987353] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [582.987394] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [582.987564] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [582.987770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [582.987861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [582.987955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [582.988045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [582.988134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [582.988221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [582.988315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [582.988404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [582.988488] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [582.988587] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [582.988707] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [582.989719] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.000293] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.000545] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.001603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.001683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.001763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.001842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.001918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.001994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.002069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.002143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.002217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.002302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.002376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.002450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.002523] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.002597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.002672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.002748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.002824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.002899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.002972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.003054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.003137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.003217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.003307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.003388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.003470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.003548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.003625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.003705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.003786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.003865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.003948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.004032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.004119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.004205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.004294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.004376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.004457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.004535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.004612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.004689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.004769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.004847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.004925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.005002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.005083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.005163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.005244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.005333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.005412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.005488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.005568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.005651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.005730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.005808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.005884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.005965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.006043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.006121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.006200] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.006319] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.006323] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52622, lrc_seqno=52622, guc_id=0, flags=0x73 in no process [-1]
<7> [583.006326] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.006385] ------------[ cut here ]------------
<4> [583.006386] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.006387] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.006459] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.006524] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.006529] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.006532] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.006533] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.006534] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.006539] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.006609] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.006611] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.006613] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.006614] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.006616] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.006617] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.006618] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.006619] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.006621] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.006622] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.006624] PKRU: 55555554
<4> [583.006625] Call Trace:
<4> [583.006626] <TASK>
<4> [583.006630] ? lock_sync+0x100/0x100
<4> [583.006634] ? lock_release+0xd0/0x2b0
<4> [583.006640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.006646] process_one_work+0x239/0x760
<4> [583.006652] worker_thread+0x200/0x3f0
<4> [583.006655] ? __pfx_worker_thread+0x10/0x10
<4> [583.006657] kthread+0x10d/0x150
<4> [583.006660] ? __pfx_kthread+0x10/0x10
<4> [583.006664] ret_from_fork+0x3d4/0x480
<4> [583.006666] ? __pfx_kthread+0x10/0x10
<4> [583.006669] ret_from_fork_asm+0x1a/0x30
<4> [583.006676] </TASK>
<4> [583.006678] irq event stamp: 209563
<4> [583.006679] hardirqs last enabled at (209569): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.006681] hardirqs last disabled at (209574): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.006684] softirqs last enabled at (208454): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.006686] softirqs last disabled at (208447): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.006688] ---[ end trace 0000000000000000 ]---
<6> [583.006690] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.006760] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.006767] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.008024] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.008128] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.008235] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.008446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.008537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.008631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.008719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.008806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.008892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.008978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.009068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.009155] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.009252] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.009381] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.010388] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.021111] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.021366] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.022427] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.022501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.022571] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.022642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.022711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.022780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.022849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.022917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.022987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.023058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.023129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.023198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.023266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.023349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.023424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.023498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.023572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.023647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.023721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.023801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.023881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.023959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.024039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.024117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.024198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.024277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.024370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.024450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.024530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.024608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.024690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.024774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.024861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.024945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.025029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.025114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.025197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.025277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.025368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.025445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.025524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.025602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.025680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.025757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.025841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.025920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.025998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.026074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.026153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.026229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.026315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.026397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.026476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.026554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.026631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.026708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.026785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.026861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.026941] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.027053] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.027057] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52622, lrc_seqno=52622, guc_id=0, flags=0x73 in no process [-1]
<7> [583.027059] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.027117] ------------[ cut here ]------------
<4> [583.027119] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.027120] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:31/12306
<4> [583.027191] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.027256] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.027261] CPU: 4 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.027264] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.027265] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.027266] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.027272] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.027355] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.027357] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.027359] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.027361] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.027362] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.027363] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.027365] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.027366] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [583.027368] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.027370] CR2: 00007ddcc54aa008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [583.027371] PKRU: 55555554
<4> [583.027373] Call Trace:
<4> [583.027374] <TASK>
<4> [583.027378] ? lock_sync+0x100/0x100
<4> [583.027382] ? lock_release+0xd0/0x2b0
<4> [583.027387] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.027393] process_one_work+0x239/0x760
<4> [583.027400] worker_thread+0x200/0x3f0
<4> [583.027402] ? __pfx_worker_thread+0x10/0x10
<4> [583.027405] kthread+0x10d/0x150
<4> [583.027408] ? __pfx_kthread+0x10/0x10
<4> [583.027412] ret_from_fork+0x3d4/0x480
<4> [583.027414] ? __pfx_kthread+0x10/0x10
<4> [583.027417] ret_from_fork_asm+0x1a/0x30
<4> [583.027425] </TASK>
<4> [583.027426] irq event stamp: 502669
<4> [583.027427] hardirqs last enabled at (502675): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.027430] hardirqs last disabled at (502680): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.027433] softirqs last enabled at (501584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.027435] softirqs last disabled at (501575): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.027438] ---[ end trace 0000000000000000 ]---
<5> [583.028018] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52677, lrc_seqno=52677, guc_id=0, flags=0x73 in no process [-1]
<7> [583.028021] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.028080] ------------[ cut here ]------------
<4> [583.028082] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.028083] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:31/12306
<4> [583.028153] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.028217] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.028221] CPU: 4 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.028223] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.028225] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.028226] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.028230] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.028303] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.028304] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.028307] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.028308] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.028309] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.028311] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.028312] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.028314] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [583.028315] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.028316] CR2: 00007ddcc54aa008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [583.028318] PKRU: 55555554
<4> [583.028319] Call Trace:
<4> [583.028320] <TASK>
<4> [583.028323] ? lock_sync+0x100/0x100
<4> [583.028327] ? lock_release+0xd0/0x2b0
<4> [583.028333] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.028338] process_one_work+0x239/0x760
<4> [583.028344] worker_thread+0x200/0x3f0
<4> [583.028347] ? __pfx_worker_thread+0x10/0x10
<4> [583.028350] kthread+0x10d/0x150
<4> [583.028352] ? __pfx_kthread+0x10/0x10
<4> [583.028356] ret_from_fork+0x3d4/0x480
<4> [583.028358] ? __pfx_kthread+0x10/0x10
<4> [583.028361] ret_from_fork_asm+0x1a/0x30
<4> [583.028369] </TASK>
<4> [583.028370] irq event stamp: 505273
<4> [583.028372] hardirqs last enabled at (505279): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.028374] hardirqs last disabled at (505284): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.028377] softirqs last enabled at (501584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.028379] softirqs last disabled at (501575): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.028381] ---[ end trace 0000000000000000 ]---
<6> [583.029181] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.029252] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.029258] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.029273] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.029388] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.029494] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.029702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.029793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.029888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.029978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.030068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.030157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.030246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.030357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.030440] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.030537] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.030654] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.031671] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.042294] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.042548] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.043612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.043693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.043771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.043849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.043930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.044009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.044087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.044164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.044238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.044325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.044399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.044473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.044546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.044620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.044695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.044772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.044848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.044924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.044999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.045081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.045163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.045242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.045332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.045412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.045491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.045572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.045651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.045733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.045813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.045892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.045972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.046052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.046141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.046239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.046389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.046505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.046614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.046717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.046827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.046929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.047035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.047139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.047226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.047312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.047399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.047481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.047562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.047641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.047723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.047803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.047882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.047963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.048042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.048120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.048197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.048275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.048366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.048444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.048527] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.048639] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.048644] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52677, lrc_seqno=52677, guc_id=0, flags=0x73 in no process [-1]
<7> [583.048646] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.048705] ------------[ cut here ]------------
<4> [583.048706] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.048707] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.048780] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.048847] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.048852] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.048855] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.048856] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.048857] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.048863] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.048934] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.048935] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.048937] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.048939] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.048940] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.048941] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.048943] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.048944] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.048946] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.048947] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.048948] PKRU: 55555554
<4> [583.048950] Call Trace:
<4> [583.048951] <TASK>
<4> [583.048955] ? lock_sync+0x100/0x100
<4> [583.048959] ? lock_release+0xd0/0x2b0
<4> [583.048965] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.048971] process_one_work+0x239/0x760
<4> [583.048977] worker_thread+0x200/0x3f0
<4> [583.048980] ? __pfx_worker_thread+0x10/0x10
<4> [583.048982] kthread+0x10d/0x150
<4> [583.048985] ? __pfx_kthread+0x10/0x10
<4> [583.048989] ret_from_fork+0x3d4/0x480
<4> [583.048991] ? __pfx_kthread+0x10/0x10
<4> [583.048994] ret_from_fork_asm+0x1a/0x30
<4> [583.049002] </TASK>
<4> [583.049003] irq event stamp: 213087
<4> [583.049004] hardirqs last enabled at (213093): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.049007] hardirqs last disabled at (213098): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.049009] softirqs last enabled at (211876): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.049011] softirqs last disabled at (211871): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.049014] ---[ end trace 0000000000000000 ]---
<6> [583.049016] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.049085] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.049091] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.049504] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.049706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.049791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.049882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.049973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.050067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.050154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.050240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.050339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.050426] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.050523] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.050641] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.051696] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.062426] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.062669] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.063672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.063754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.063829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.063901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.063970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.064041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.064110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.064181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.064251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.064335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.064411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.064485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.064559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.064633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.064708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.064781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.064854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.064929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.065004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.065084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.065166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.065245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.065339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.065419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.065497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.065575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.065651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.065732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.065813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.065892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.065971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.066050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.066133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.066217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.066309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.066389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.066469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.066549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.066627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.066706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.066785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.066863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.066939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.067015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.067096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.067173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.067249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.067337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.067421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.067501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.067579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.067659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.067737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.067815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.067892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.067970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.068046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.068121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.068199] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.068315] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.068319] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52677, lrc_seqno=52677, guc_id=0, flags=0x73 in no process [-1]
<7> [583.068322] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.068380] ------------[ cut here ]------------
<4> [583.068382] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.068383] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.068454] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.068520] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.068525] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.068528] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.068529] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.068531] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.068536] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.068606] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.068608] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.068610] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.068612] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.068613] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.068614] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.068615] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.068617] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.068618] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.068620] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.068621] PKRU: 55555554
<4> [583.068622] Call Trace:
<4> [583.068623] <TASK>
<4> [583.068628] ? lock_sync+0x100/0x100
<4> [583.068632] ? lock_release+0xd0/0x2b0
<4> [583.068638] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.068644] process_one_work+0x239/0x760
<4> [583.068650] worker_thread+0x200/0x3f0
<4> [583.068653] ? __pfx_worker_thread+0x10/0x10
<4> [583.068655] kthread+0x10d/0x150
<4> [583.068658] ? __pfx_kthread+0x10/0x10
<4> [583.068662] ret_from_fork+0x3d4/0x480
<4> [583.068664] ? __pfx_kthread+0x10/0x10
<4> [583.068667] ret_from_fork_asm+0x1a/0x30
<4> [583.068674] </TASK>
<4> [583.068676] irq event stamp: 216225
<4> [583.068677] hardirqs last enabled at (216231): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.068679] hardirqs last disabled at (216236): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.068682] softirqs last enabled at (215206): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.068684] softirqs last disabled at (215199): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.068686] ---[ end trace 0000000000000000 ]---
<5> [583.069273] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52732, lrc_seqno=52732, guc_id=0, flags=0x73 in no process [-1]
<7> [583.069276] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.070322] ------------[ cut here ]------------
<4> [583.070323] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.070325] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.070406] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.070470] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.070475] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.070477] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.070478] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.070480] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.070484] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.070556] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.070557] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.070559] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.070561] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.070562] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.070563] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.070564] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.070566] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.070567] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.070568] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.070570] PKRU: 55555554
<4> [583.070571] Call Trace:
<4> [583.070572] <TASK>
<4> [583.070576] ? lock_sync+0x100/0x100
<4> [583.070580] ? lock_release+0xd0/0x2b0
<4> [583.070585] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.070591] process_one_work+0x239/0x760
<4> [583.070597] worker_thread+0x200/0x3f0
<4> [583.070599] ? __pfx_worker_thread+0x10/0x10
<4> [583.070602] kthread+0x10d/0x150
<4> [583.070605] ? __pfx_kthread+0x10/0x10
<4> [583.070608] ret_from_fork+0x3d4/0x480
<4> [583.070610] ? __pfx_kthread+0x10/0x10
<4> [583.070614] ret_from_fork_asm+0x1a/0x30
<4> [583.070621] </TASK>
<4> [583.070622] irq event stamp: 218957
<4> [583.070623] hardirqs last enabled at (218963): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.070625] hardirqs last disabled at (218968): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.070628] softirqs last enabled at (218184): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.070630] softirqs last disabled at (218173): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.070632] ---[ end trace 0000000000000000 ]---
<6> [583.070634] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.070702] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.070708] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.070723] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.070827] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.070933] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.071138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.071231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.071332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.071420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.071507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.071592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.071677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.071765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.071848] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.071942] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.072059] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.073064] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.083291] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.083542] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.084641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.084722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.084799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.084876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.084954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.085030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.085107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.085183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.085259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.085344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.085420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.085496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.085574] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.085650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.085725] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.085800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.085875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.085950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.086026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.086108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.086189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.086268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.086360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.086439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.086519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.086601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.086681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.086762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.086841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.086919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.086998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.087075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.087161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.087245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.087337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.087418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.087500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.087582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.087662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.087740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.087820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.087899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.087978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.088055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.088136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.088213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.088294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.088369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.088447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.088522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.088603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.088686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.088766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.088845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.088922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.089001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.089077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.089157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.089242] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.089363] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.089367] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52732, lrc_seqno=52732, guc_id=0, flags=0x73 in no process [-1]
<7> [583.089369] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.089427] ------------[ cut here ]------------
<4> [583.089428] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.089429] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:31/12306
<4> [583.089500] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.089565] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.089570] CPU: 4 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.089572] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.089574] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.089575] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.089580] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.089649] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.089650] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.089652] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.089654] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.089655] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.089656] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.089657] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.089659] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [583.089660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.089662] CR2: 00007ddcc54aa008 CR3: 000000011ea25002 CR4: 0000000000f72ef0
<4> [583.089663] PKRU: 55555554
<4> [583.089665] Call Trace:
<4> [583.089666] <TASK>
<4> [583.089669] ? lock_sync+0x100/0x100
<4> [583.089674] ? lock_release+0xd0/0x2b0
<4> [583.089679] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.089685] process_one_work+0x239/0x760
<4> [583.089691] worker_thread+0x200/0x3f0
<4> [583.089693] ? __pfx_worker_thread+0x10/0x10
<4> [583.089696] kthread+0x10d/0x150
<4> [583.089699] ? __pfx_kthread+0x10/0x10
<4> [583.089702] ret_from_fork+0x3d4/0x480
<4> [583.089704] ? __pfx_kthread+0x10/0x10
<4> [583.089708] ret_from_fork_asm+0x1a/0x30
<4> [583.089715] </TASK>
<4> [583.089716] irq event stamp: 510953
<4> [583.089717] hardirqs last enabled at (510959): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.089720] hardirqs last disabled at (510964): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.089722] softirqs last enabled at (509740): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.089725] softirqs last disabled at (509731): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.089727] ---[ end trace 0000000000000000 ]---
<6> [583.089728] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.089798] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.089803] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.091051] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.091155] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.091262] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.091489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.091582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.091675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.091763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.091849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.091934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.092020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.092106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.092187] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.092291] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.092410] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.093416] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.103293] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.103543] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.104631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.104711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.104788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.104867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.104947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.105025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.105102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.105178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.105252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.105336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.105411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.105484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.105558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.105633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.105708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.105782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.105856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.105930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.106004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.106084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.106166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.106248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.106339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.106417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.106496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.106574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.106651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.106731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.106811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.106890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.106970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.107048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.107132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.107215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.107301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.107382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.107462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.107540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.107621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.107703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.107785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.107864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.107943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.108020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.108103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.108180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.108258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.108346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.108424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.108500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.108576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.108655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.108735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.108816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.108895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.108975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.109050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.109126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.109204] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.109319] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.109323] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52732, lrc_seqno=52732, guc_id=0, flags=0x73 in no process [-1]
<7> [583.109326] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.109385] ------------[ cut here ]------------
<4> [583.109386] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.109387] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [583.109459] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.109524] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.109529] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.109532] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.109533] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.109535] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.109540] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.109609] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.109610] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [583.109613] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.109614] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.109615] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.109617] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.109618] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.109619] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [583.109621] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.109622] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [583.109624] PKRU: 55555554
<4> [583.109625] Call Trace:
<4> [583.109626] <TASK>
<4> [583.109630] ? lock_sync+0x100/0x100
<4> [583.109634] ? lock_release+0xd0/0x2b0
<4> [583.109640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.109645] process_one_work+0x239/0x760
<4> [583.109651] worker_thread+0x200/0x3f0
<4> [583.109654] ? __pfx_worker_thread+0x10/0x10
<4> [583.109657] kthread+0x10d/0x150
<4> [583.109660] ? __pfx_kthread+0x10/0x10
<4> [583.109663] ret_from_fork+0x3d4/0x480
<4> [583.109665] ? __pfx_kthread+0x10/0x10
<4> [583.109669] ret_from_fork_asm+0x1a/0x30
<4> [583.109676] </TASK>
<4> [583.109677] irq event stamp: 988263
<4> [583.109679] hardirqs last enabled at (988269): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.109681] hardirqs last disabled at (988274): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.109684] softirqs last enabled at (987154): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.109686] softirqs last disabled at (987145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.109688] ---[ end trace 0000000000000000 ]---
<5> [583.111192] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52787, lrc_seqno=52787, guc_id=0, flags=0x73 in no process [-1]
<7> [583.111195] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.111257] ------------[ cut here ]------------
<4> [583.111258] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.111260] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:84/12359
<4> [583.111340] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.111406] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.111411] CPU: 0 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.111413] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.111414] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.111416] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.111420] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.111490] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.111492] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [583.111494] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.111495] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.111496] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.111498] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.111499] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.111500] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [583.111502] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.111503] CR2: 00007ca9980a5050 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [583.111504] PKRU: 55555554
<4> [583.111505] Call Trace:
<4> [583.111507] <TASK>
<4> [583.111511] ? lock_sync+0x100/0x100
<4> [583.111515] ? lock_release+0xd0/0x2b0
<4> [583.111520] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.111526] process_one_work+0x239/0x760
<4> [583.111532] worker_thread+0x200/0x3f0
<4> [583.111534] ? __pfx_worker_thread+0x10/0x10
<4> [583.111537] kthread+0x10d/0x150
<4> [583.111540] ? __pfx_kthread+0x10/0x10
<4> [583.111543] ret_from_fork+0x3d4/0x480
<4> [583.111545] ? __pfx_kthread+0x10/0x10
<4> [583.111548] ret_from_fork_asm+0x1a/0x30
<4> [583.111556] </TASK>
<4> [583.111557] irq event stamp: 991009
<4> [583.111558] hardirqs last enabled at (991015): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.111561] hardirqs last disabled at (991020): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.111563] softirqs last enabled at (987154): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.111565] softirqs last disabled at (987145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.111568] ---[ end trace 0000000000000000 ]---
<7> [583.111581] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.111683] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [583.111782] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.111855] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.111861] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.111910] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.112119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.112211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.112309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.112399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.112488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.112576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.112664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.112752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.112836] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.112930] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.113046] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.114054] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.124294] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.124547] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.125600] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.125681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.125758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.125837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.125918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.125996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.126072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.126147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.126222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.126336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.126412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.126486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.126560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.126635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.126709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.126783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.126860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.126938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.127013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.127097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.127180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.127260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.127355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.127438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.127518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.127596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.127674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.127755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.127836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.127915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.127995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.128075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.128161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.128246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.128340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.128421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.128501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.128579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.128656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.128733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.128813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.128892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.128970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.129047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.129128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.129205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.129283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.129374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.129453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.129528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.129605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.129684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.129760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.129842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.129924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.130006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.130084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.130161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.130241] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.130365] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.130369] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52787, lrc_seqno=52787, guc_id=0, flags=0x73 in no process [-1]
<7> [583.130372] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.130430] ------------[ cut here ]------------
<4> [583.130432] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.130433] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.130512] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.130586] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.130591] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.130594] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.130595] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.130597] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.130602] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.130674] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.130675] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.130678] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.130680] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.130681] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.130683] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.130684] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.130686] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.130687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.130689] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.130691] PKRU: 55555554
<4> [583.130692] Call Trace:
<4> [583.130694] <TASK>
<4> [583.130698] ? lock_sync+0x100/0x100
<4> [583.130703] ? lock_release+0xd0/0x2b0
<4> [583.130708] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.130714] process_one_work+0x239/0x760
<4> [583.130720] worker_thread+0x200/0x3f0
<4> [583.130723] ? __pfx_worker_thread+0x10/0x10
<4> [583.130726] kthread+0x10d/0x150
<4> [583.130729] ? __pfx_kthread+0x10/0x10
<4> [583.130733] ret_from_fork+0x3d4/0x480
<4> [583.130735] ? __pfx_kthread+0x10/0x10
<4> [583.130738] ret_from_fork_asm+0x1a/0x30
<4> [583.130745] </TASK>
<4> [583.130747] irq event stamp: 224527
<4> [583.130748] hardirqs last enabled at (224533): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.130751] hardirqs last disabled at (224538): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.130753] softirqs last enabled at (223418): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.130755] softirqs last disabled at (223409): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.130758] ---[ end trace 0000000000000000 ]---
<6> [583.130760] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.130829] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.130835] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.131245] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.131508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.131598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.131693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.131784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.131873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.131959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.132046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.132133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.132215] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.132317] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.132435] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.133441] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.143296] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.143549] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.144623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.144704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.144782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.144859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.144938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.145017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.145095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.145170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.145245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.145331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.145405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.145479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.145553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.145627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.145701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.145775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.145853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.145930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.146006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.146090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.146178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.146258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.146350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.146429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.146507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.146586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.146663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.146744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.146825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.146907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.146991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.147073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.147158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.147243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.147344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.147426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.147507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.147586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.147663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.147739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.147818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.147896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.147973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.148049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.148129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.148206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.148282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.148375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.148453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.148530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.148605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.148684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.148762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.148842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.148920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.149000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.149077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.149154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.149233] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.149352] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.149356] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52787, lrc_seqno=52787, guc_id=0, flags=0x73 in no process [-1]
<7> [583.149358] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.149418] ------------[ cut here ]------------
<4> [583.149419] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.149420] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.149493] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.149557] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.149562] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.149565] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.149566] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.149567] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.149572] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.149642] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.149644] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.149646] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.149647] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.149648] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.149650] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.149651] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.149652] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.149654] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.149655] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.149657] PKRU: 55555554
<4> [583.149658] Call Trace:
<4> [583.149659] <TASK>
<4> [583.149663] ? lock_sync+0x100/0x100
<4> [583.149667] ? lock_release+0xd0/0x2b0
<4> [583.149673] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.149679] process_one_work+0x239/0x760
<4> [583.149685] worker_thread+0x200/0x3f0
<4> [583.149687] ? __pfx_worker_thread+0x10/0x10
<4> [583.149690] kthread+0x10d/0x150
<4> [583.149693] ? __pfx_kthread+0x10/0x10
<4> [583.149696] ret_from_fork+0x3d4/0x480
<4> [583.149698] ? __pfx_kthread+0x10/0x10
<4> [583.149702] ret_from_fork_asm+0x1a/0x30
<4> [583.149709] </TASK>
<4> [583.149710] irq event stamp: 227663
<4> [583.149711] hardirqs last enabled at (227669): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.149714] hardirqs last disabled at (227674): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.149716] softirqs last enabled at (226742): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.149719] softirqs last disabled at (226735): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.149721] ---[ end trace 0000000000000000 ]---
<5> [583.151239] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52842, lrc_seqno=52842, guc_id=0, flags=0x73 in no process [-1]
<7> [583.151242] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.151322] ------------[ cut here ]------------
<4> [583.151324] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.151325] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:54/12329
<4> [583.151396] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.151460] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.151464] CPU: 11 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.151467] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.151468] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.151469] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.151474] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.151542] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.151543] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.151546] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.151547] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.151548] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.151549] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.151550] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.151552] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [583.151553] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.151555] CR2: 0000751847a00088 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [583.151556] PKRU: 55555554
<4> [583.151557] Call Trace:
<4> [583.151558] <TASK>
<4> [583.151562] ? lock_sync+0x100/0x100
<4> [583.151566] ? lock_release+0xd0/0x2b0
<4> [583.151572] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.151577] process_one_work+0x239/0x760
<4> [583.151583] worker_thread+0x200/0x3f0
<4> [583.151586] ? __pfx_worker_thread+0x10/0x10
<4> [583.151588] kthread+0x10d/0x150
<4> [583.151591] ? __pfx_kthread+0x10/0x10
<4> [583.151594] ret_from_fork+0x3d4/0x480
<4> [583.151596] ? __pfx_kthread+0x10/0x10
<4> [583.151600] ret_from_fork_asm+0x1a/0x30
<4> [583.151607] </TASK>
<4> [583.151608] irq event stamp: 230231
<4> [583.151609] hardirqs last enabled at (230237): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.151612] hardirqs last disabled at (230242): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.151614] softirqs last enabled at (226742): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.151617] softirqs last disabled at (226735): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.151619] ---[ end trace 0000000000000000 ]---
<6> [583.151621] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.151690] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.151697] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.151712] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.151816] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.151927] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.152135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.152228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.152329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.152421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.152511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.152598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.152688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.152778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.152863] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.152959] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.153076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.154089] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.164296] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.164545] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.165634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.165712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.165787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.165865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.165943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.166023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.166102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.166179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.166254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.166339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.166414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.166489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.166564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.166638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.166711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.166785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.166858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.166932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.167005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.167087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.167171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.167251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.167340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.167419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.167496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.167573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.167650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.167729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.167809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.167892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.167974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.168055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.168141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.168225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.168315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.168397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.168482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.168564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.168643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.168720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.168801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.168879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.168957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.169035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.169119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.169200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.169280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.169372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.169451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.169527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.169603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.169683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.169761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.169839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.169915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.169993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.170069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.170145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.170225] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.170341] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.170345] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52842, lrc_seqno=52842, guc_id=0, flags=0x73 in no process [-1]
<7> [583.170348] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.170407] ------------[ cut here ]------------
<4> [583.170408] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.170409] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:84/12359
<4> [583.170481] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.170545] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.170550] CPU: 2 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.170553] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.170554] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.170556] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.170561] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.170632] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.170633] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [583.170636] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.170637] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.170638] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.170639] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.170641] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.170642] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [583.170644] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.170645] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [583.170647] PKRU: 55555554
<4> [583.170648] Call Trace:
<4> [583.170649] <TASK>
<4> [583.170653] ? lock_sync+0x100/0x100
<4> [583.170657] ? lock_release+0xd0/0x2b0
<4> [583.170663] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.170668] process_one_work+0x239/0x760
<4> [583.170674] worker_thread+0x200/0x3f0
<4> [583.170677] ? __pfx_worker_thread+0x10/0x10
<4> [583.170680] kthread+0x10d/0x150
<4> [583.170682] ? __pfx_kthread+0x10/0x10
<4> [583.170686] ret_from_fork+0x3d4/0x480
<4> [583.170688] ? __pfx_kthread+0x10/0x10
<4> [583.170691] ret_from_fork_asm+0x1a/0x30
<4> [583.170699] </TASK>
<4> [583.170700] irq event stamp: 996391
<4> [583.170701] hardirqs last enabled at (996397): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.170704] hardirqs last disabled at (996402): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.170706] softirqs last enabled at (995470): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.170708] softirqs last disabled at (995463): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.170711] ---[ end trace 0000000000000000 ]---
<6> [583.170713] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.170783] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.170790] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.172033] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.172140] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.172248] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.172466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.172560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.172660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.172751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.172840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.172928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.173015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.173104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.173188] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.173284] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.173417] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.174425] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.185149] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.185394] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.186461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.186542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.186619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.186697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.186774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.186851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.186929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.187007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.187083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.187159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.187234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.187315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.187390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.187464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.187537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.187612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.187686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.187761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.187834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.187918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.188001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.188082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.188161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.188240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.188325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.188402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.188483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.188566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.188648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.188728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.188808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.188887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.188972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.189056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.189146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.189230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.189319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.189400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.189489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.189565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.189646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.189724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.189802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.189879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.189964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.190045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.190125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.190203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.190282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.190373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.190451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.190535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.190615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.190694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.190772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.190852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.190929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.191004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.191083] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.191195] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.191200] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52842, lrc_seqno=52842, guc_id=0, flags=0x73 in no process [-1]
<7> [583.191202] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.191262] ------------[ cut here ]------------
<4> [583.191263] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.191264] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:31/12306
<4> [583.191346] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.191413] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.191418] CPU: 6 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.191421] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.191422] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.191424] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.191429] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.191501] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.191502] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.191505] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.191506] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.191508] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.191509] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.191510] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.191511] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [583.191513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.191514] CR2: 00007ddcc54ac008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [583.191516] PKRU: 55555554
<4> [583.191517] Call Trace:
<4> [583.191518] <TASK>
<4> [583.191522] ? lock_sync+0x100/0x100
<4> [583.191527] ? lock_release+0xd0/0x2b0
<4> [583.191532] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.191538] process_one_work+0x239/0x760
<4> [583.191544] worker_thread+0x200/0x3f0
<4> [583.191547] ? __pfx_worker_thread+0x10/0x10
<4> [583.191549] kthread+0x10d/0x150
<4> [583.191552] ? __pfx_kthread+0x10/0x10
<4> [583.191556] ret_from_fork+0x3d4/0x480
<4> [583.191558] ? __pfx_kthread+0x10/0x10
<4> [583.191561] ret_from_fork_asm+0x1a/0x30
<4> [583.191568] </TASK>
<4> [583.191570] irq event stamp: 514799
<4> [583.191571] hardirqs last enabled at (514805): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.191574] hardirqs last disabled at (514810): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.191576] softirqs last enabled at (513796): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.191578] softirqs last disabled at (513791): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.191581] ---[ end trace 0000000000000000 ]---
<5> [583.193084] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52897, lrc_seqno=52897, guc_id=0, flags=0x73 in no process [-1]
<7> [583.193087] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.193149] ------------[ cut here ]------------
<4> [583.193150] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.193152] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:31/12306
<4> [583.193221] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.193284] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.193293] CPU: 6 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.193296] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.193297] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.193298] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.193303] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.193371] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.193373] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.193375] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.193376] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.193377] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.193378] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.193380] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.193381] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [583.193382] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.193384] CR2: 00007ddcc54ac008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [583.193385] PKRU: 55555554
<4> [583.193386] Call Trace:
<4> [583.193387] <TASK>
<4> [583.193391] ? lock_sync+0x100/0x100
<4> [583.193395] ? lock_release+0xd0/0x2b0
<4> [583.193401] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.193406] process_one_work+0x239/0x760
<4> [583.193412] worker_thread+0x200/0x3f0
<4> [583.193415] ? __pfx_worker_thread+0x10/0x10
<4> [583.193417] kthread+0x10d/0x150
<4> [583.193420] ? __pfx_kthread+0x10/0x10
<4> [583.193423] ret_from_fork+0x3d4/0x480
<4> [583.193425] ? __pfx_kthread+0x10/0x10
<4> [583.193429] ret_from_fork_asm+0x1a/0x30
<4> [583.193436] </TASK>
<4> [583.193437] irq event stamp: 517505
<4> [583.193438] hardirqs last enabled at (517511): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.193441] hardirqs last disabled at (517516): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.193443] softirqs last enabled at (513796): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.193445] softirqs last disabled at (513791): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.193447] ---[ end trace 0000000000000000 ]---
<7> [583.193462] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.193566] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [583.193665] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.193739] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.193745] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.193791] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.193995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.194086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.194178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.194267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.194365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.194452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.194540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.194632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.194719] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.194817] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.194935] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.195943] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.206294] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.206544] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.207606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.207684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.207760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.207840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.207920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.207998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.208075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.208151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.208228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.208308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.208385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.208460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.208538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.208617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.208693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.208769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.208844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.208919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.208993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.209075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.209155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.209234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.209321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.209400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.209478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.209557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.209638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.209721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.209802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.209882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.209961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.210045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.210133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.210219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.210308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.210390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.210471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.210549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.210627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.210702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.210781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.210858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.210939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.211019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.211102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.211181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.211260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.211345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.211424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.211500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.211574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.211652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.211732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.211813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.211891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.211970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.212046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.212122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.212201] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.212317] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.212321] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52897, lrc_seqno=52897, guc_id=0, flags=0x73 in no process [-1]
<7> [583.212323] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.212381] ------------[ cut here ]------------
<4> [583.212383] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.212384] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:31/12306
<4> [583.212454] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.212519] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.212523] CPU: 6 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.212526] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.212527] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.212528] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.212533] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.212603] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.212604] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.212607] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.212608] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.212609] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.212610] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.212612] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.212613] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [583.212615] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.212616] CR2: 00007ddcc54ac008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [583.212617] PKRU: 55555554
<4> [583.212619] Call Trace:
<4> [583.212620] <TASK>
<4> [583.212623] ? lock_sync+0x100/0x100
<4> [583.212628] ? lock_release+0xd0/0x2b0
<4> [583.212633] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.212639] process_one_work+0x239/0x760
<4> [583.212645] worker_thread+0x200/0x3f0
<4> [583.212648] ? __pfx_worker_thread+0x10/0x10
<4> [583.212650] kthread+0x10d/0x150
<4> [583.212653] ? __pfx_kthread+0x10/0x10
<4> [583.212657] ret_from_fork+0x3d4/0x480
<4> [583.212659] ? __pfx_kthread+0x10/0x10
<4> [583.212662] ret_from_fork_asm+0x1a/0x30
<4> [583.212670] </TASK>
<4> [583.212671] irq event stamp: 520633
<4> [583.212672] hardirqs last enabled at (520639): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.212675] hardirqs last disabled at (520644): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.212677] softirqs last enabled at (519614): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.212679] softirqs last disabled at (519607): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.212682] ---[ end trace 0000000000000000 ]---
<6> [583.212683] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.212753] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.212759] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.213996] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.214103] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.214210] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.214419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.214510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.214603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.214692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.214779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.214865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.214952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.215040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.215123] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.215219] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.215345] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.216350] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.226295] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.226547] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.227623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.227707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.227787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.227865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.227941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.228017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.228093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.228169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.228246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.228338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.228414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.228489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.228563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.228639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.228717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.228796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.228873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.228949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.229024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.229105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.229186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.229266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.229357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.229435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.229514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.229592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.229673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.229757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.229840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.229921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.230001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.230082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.230168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.230253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.230345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.230428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.230512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.230592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.230671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.230748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.230827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.230906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.230987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.231067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.231149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.231228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.231309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.231387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.231465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.231543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.231622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.231704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.231783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.231861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.231938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.232016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.232092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.232169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.232249] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.232371] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.232375] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52897, lrc_seqno=52897, guc_id=0, flags=0x73 in no process [-1]
<7> [583.232377] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.232435] ------------[ cut here ]------------
<4> [583.232436] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.232438] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:84/12359
<4> [583.232509] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.232574] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.232578] CPU: 2 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.232581] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.232582] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.232583] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.232589] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.232659] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.232661] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [583.232663] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.232665] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.232666] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.232667] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.232668] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.232670] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [583.232671] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.232673] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [583.232674] PKRU: 55555554
<4> [583.232676] Call Trace:
<4> [583.232677] <TASK>
<4> [583.232681] ? lock_sync+0x100/0x100
<4> [583.232685] ? lock_release+0xd0/0x2b0
<4> [583.232691] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.232696] process_one_work+0x239/0x760
<4> [583.232702] worker_thread+0x200/0x3f0
<4> [583.232705] ? __pfx_worker_thread+0x10/0x10
<4> [583.232708] kthread+0x10d/0x150
<4> [583.232710] ? __pfx_kthread+0x10/0x10
<4> [583.232714] ret_from_fork+0x3d4/0x480
<4> [583.232716] ? __pfx_kthread+0x10/0x10
<4> [583.232719] ret_from_fork_asm+0x1a/0x30
<4> [583.232727] </TASK>
<4> [583.232728] irq event stamp: 999971
<4> [583.232729] hardirqs last enabled at (999977): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.232732] hardirqs last disabled at (999982): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.232734] softirqs last enabled at (999044): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.232736] softirqs last disabled at (999037): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.232738] ---[ end trace 0000000000000000 ]---
<5> [583.234237] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52952, lrc_seqno=52952, guc_id=0, flags=0x73 in no process [-1]
<7> [583.234240] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.234313] ------------[ cut here ]------------
<4> [583.234315] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.234316] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:84/12359
<4> [583.234387] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.234451] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.234455] CPU: 2 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.234458] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.234459] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.234460] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.234465] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.234534] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.234535] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [583.234537] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.234538] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.234540] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.234541] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.234542] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.234543] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [583.234545] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.234546] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [583.234547] PKRU: 55555554
<4> [583.234549] Call Trace:
<4> [583.234550] <TASK>
<4> [583.234553] ? lock_sync+0x100/0x100
<4> [583.234558] ? lock_release+0xd0/0x2b0
<4> [583.234563] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.234568] process_one_work+0x239/0x760
<4> [583.234574] worker_thread+0x200/0x3f0
<4> [583.234577] ? __pfx_worker_thread+0x10/0x10
<4> [583.234579] kthread+0x10d/0x150
<4> [583.234582] ? __pfx_kthread+0x10/0x10
<4> [583.234586] ret_from_fork+0x3d4/0x480
<4> [583.234587] ? __pfx_kthread+0x10/0x10
<4> [583.234591] ret_from_fork_asm+0x1a/0x30
<4> [583.234598] </TASK>
<4> [583.234599] irq event stamp: 1002733
<4> [583.234601] hardirqs last enabled at (1002739): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.234603] hardirqs last disabled at (1002744): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.234605] softirqs last enabled at (1001966): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.234608] softirqs last disabled at (1001955): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.234610] ---[ end trace 0000000000000000 ]---
<6> [583.234612] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.234681] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.234686] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.234701] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.234805] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.234908] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.235114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.235204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.235339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.235427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.235516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.235603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.235691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.235779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.235862] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.235957] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.236074] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.237081] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.247298] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.247550] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.248621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.248702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.248779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.248857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.248936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.249014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.249090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.249167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.249245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.249330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.249406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.249481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.249557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.249631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.249705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.249782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.249857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.249933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.250007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.250089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.250170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.250249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.250339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.250421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.250500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.250578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.250656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.250736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.250816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.250895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.250977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.251059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.251145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.251230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.251317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.251400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.251480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.251558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.251636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.251712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.251792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.251870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.251947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.252024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.252104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.252181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.252258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.252342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.252422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.252501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.252580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.252661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.252738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.252816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.252892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.252969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.253046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.253126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.253206] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.253320] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.253325] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52952, lrc_seqno=52952, guc_id=0, flags=0x73 in no process [-1]
<7> [583.253327] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.253386] ------------[ cut here ]------------
<4> [583.253387] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.253388] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:31/12306
<4> [583.253460] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.253525] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.253529] CPU: 6 UID: 0 PID: 12306 Comm: kworker/u64:31 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.253532] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.253533] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.253535] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.253540] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.253609] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.253611] RSP: 0018:ffffc9000121bca0 EFLAGS: 00010246
<4> [583.253613] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.253614] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.253616] RBP: ffffc9000121bdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.253617] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.253618] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.253619] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [583.253621] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.253622] CR2: 00007ddcc54ac008 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [583.253624] PKRU: 55555554
<4> [583.253625] Call Trace:
<4> [583.253626] <TASK>
<4> [583.253630] ? lock_sync+0x100/0x100
<4> [583.253635] ? lock_release+0xd0/0x2b0
<4> [583.253640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.253646] process_one_work+0x239/0x760
<4> [583.253652] worker_thread+0x200/0x3f0
<4> [583.253655] ? __pfx_worker_thread+0x10/0x10
<4> [583.253657] kthread+0x10d/0x150
<4> [583.253660] ? __pfx_kthread+0x10/0x10
<4> [583.253664] ret_from_fork+0x3d4/0x480
<4> [583.253666] ? __pfx_kthread+0x10/0x10
<4> [583.253669] ret_from_fork_asm+0x1a/0x30
<4> [583.253676] </TASK>
<4> [583.253677] irq event stamp: 526437
<4> [583.253679] hardirqs last enabled at (526443): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.253681] hardirqs last disabled at (526448): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.253683] softirqs last enabled at (525328): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.253686] softirqs last disabled at (525321): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.253688] ---[ end trace 0000000000000000 ]---
<6> [583.253690] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.253760] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.253767] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.254996] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.255099] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.255205] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.255428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.255528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.255628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.255721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.255812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.255904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.255995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.256088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.256179] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.256282] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.256419] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.257431] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.267297] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.267554] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.268550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.268634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.268716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.268799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.268883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.268965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.269046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.269127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.269206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.269285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.269379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.269462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.269544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.269626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.269706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.269785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.269865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.269948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.270027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.270114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.270200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.270284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.270391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.270478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.270561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.270644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.270725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.270809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.270895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.270978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.271061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.271145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.271235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.271330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.271419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.271504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.271593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.271680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.271766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.271850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.271935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.272021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.272105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.272186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.272273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.272370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.272456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.272539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.272623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.272706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.272786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.272871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.272956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.273042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.273128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.273212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.273297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.273379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.273465] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.273582] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.273587] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=52952, lrc_seqno=52952, guc_id=0, flags=0x73 in no process [-1]
<7> [583.273589] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.273651] ------------[ cut here ]------------
<4> [583.273653] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.273654] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [583.273731] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.273797] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.273802] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.273805] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.273807] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.273808] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.273813] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.273888] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.273889] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.273892] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.273893] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.273895] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.273896] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.273897] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.273899] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [583.273900] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.273902] CR2: 00007ddcc54ad008 CR3: 000000011ea25002 CR4: 0000000000f72ef0
<4> [583.273903] PKRU: 55555554
<4> [583.273904] Call Trace:
<4> [583.273905] <TASK>
<4> [583.273910] ? lock_sync+0x100/0x100
<4> [583.273914] ? lock_release+0xd0/0x2b0
<4> [583.273920] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.273926] process_one_work+0x239/0x760
<4> [583.273932] worker_thread+0x200/0x3f0
<4> [583.273935] ? __pfx_worker_thread+0x10/0x10
<4> [583.273938] kthread+0x10d/0x150
<4> [583.273940] ? __pfx_kthread+0x10/0x10
<4> [583.273944] ret_from_fork+0x3d4/0x480
<4> [583.273946] ? __pfx_kthread+0x10/0x10
<4> [583.273950] ret_from_fork_asm+0x1a/0x30
<4> [583.273957] </TASK>
<4> [583.273958] irq event stamp: 236043
<4> [583.273959] hardirqs last enabled at (236049): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.273962] hardirqs last disabled at (236054): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.273965] softirqs last enabled at (234934): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.273967] softirqs last disabled at (234889): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.273969] ---[ end trace 0000000000000000 ]---
<5> [583.275447] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53007, lrc_seqno=53007, guc_id=0, flags=0x73 in no process [-1]
<7> [583.275450] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.275516] ------------[ cut here ]------------
<4> [583.275517] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.275519] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:54/12329
<4> [583.275593] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.275656] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.275661] CPU: 4 UID: 0 PID: 12329 Comm: kworker/u64:54 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.275663] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.275664] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.275666] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.275670] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.275744] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.275746] RSP: 0018:ffffc90007c4fca0 EFLAGS: 00010246
<4> [583.275748] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.275749] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.275750] RBP: ffffc90007c4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.275752] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.275753] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.275754] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [583.275756] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.275757] CR2: 00007ddcc54ad008 CR3: 000000011ea25002 CR4: 0000000000f72ef0
<4> [583.275758] PKRU: 55555554
<4> [583.275760] Call Trace:
<4> [583.275761] <TASK>
<4> [583.275764] ? lock_sync+0x100/0x100
<4> [583.275769] ? lock_release+0xd0/0x2b0
<4> [583.275774] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.275780] process_one_work+0x239/0x760
<4> [583.275786] worker_thread+0x200/0x3f0
<4> [583.275789] ? __pfx_worker_thread+0x10/0x10
<4> [583.275791] kthread+0x10d/0x150
<4> [583.275794] ? __pfx_kthread+0x10/0x10
<4> [583.275797] ret_from_fork+0x3d4/0x480
<4> [583.275799] ? __pfx_kthread+0x10/0x10
<4> [583.275803] ret_from_fork_asm+0x1a/0x30
<4> [583.275810] </TASK>
<4> [583.275811] irq event stamp: 238703
<4> [583.275812] hardirqs last enabled at (238709): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.275815] hardirqs last disabled at (238714): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.275817] softirqs last enabled at (236958): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.275820] softirqs last disabled at (236935): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.275822] ---[ end trace 0000000000000000 ]---
<6> [583.275824] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.275897] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.275903] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.275918] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [583.276029] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [583.276142] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.276350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.276439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.276534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.276625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.276713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.276799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.276887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.276975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.277057] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.277153] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.277270] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.278287] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.288296] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [583.288547] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [583.289640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [583.289719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [583.289795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [583.289872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [583.289949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [583.290025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [583.290101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [583.290176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [583.290252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [583.290336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [583.290412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [583.290488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [583.290566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [583.290644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [583.290720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [583.290795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [583.290871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [583.290944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [583.291018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [583.291100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [583.291181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [583.291262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [583.291355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [583.291435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [583.291513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [583.291591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [583.291666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [583.291746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [583.291825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [583.291903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [583.291982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [583.292060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [583.292143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [583.292231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [583.292323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [583.292408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [583.292488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [583.292566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [583.292644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [583.292722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [583.292805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [583.292887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [583.292968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [583.293045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [583.293127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [583.293204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [583.293282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [583.293372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [583.293451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [583.293526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [583.293601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [583.293679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [583.293756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [583.293833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [583.293908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [583.293985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [583.294060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [583.294135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [583.294219] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [583.294333] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [583.294338] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53007, lrc_seqno=53007, guc_id=0, flags=0x73 in no process [-1]
<7> [583.294340] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [583.294398] ------------[ cut here ]------------
<4> [583.294399] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [583.294401] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:84/12359
<4> [583.294472] Modules linked in: xe snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay binfmt_misc intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic intel_powerclamp cmdlinepart eeepc_wmi spi_nor mei_pxp asus_wmi mei_hdcp mtd sparse_keymap coretemp platform_profile wmi_bmof kvm_intel kvm snd_intel_dspcfg usbhid snd_hda_codec irqbypass hid ghash_clmulni_intel snd_hda_core aesni_intel r8169 snd_hwdep rapl intel_cstate snd_pcm video realtek nls_iso8859_1 snd_timer i2c_i801 snd i2c_mux spi_intel_pci idma64 mei_me i2c_smbus spi_intel soundcore mei intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [583.294537] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [583.294541] CPU: 2 UID: 0 PID: 12359 Comm: kworker/u64:84 Tainted: G S U W 7.0.0-lgci-xe-xe-4908-b2dbef367fba940bd-debug+ #1 PREEMPT(lazy)
<4> [583.294544] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [583.294545] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [583.294546] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [583.294552] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [583.294621] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [583.294623] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [583.294625] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [583.294626] RDX: ffff888104082010 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [583.294628] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [583.294629] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [583.294630] R13: ffff888104082010 R14: ffff888150a40818 R15: 00000000ffffffc2
<4> [583.294631] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [583.294633] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [583.294634] CR2: 00007bda80284058 CR3: 0000000124a85005 CR4: 0000000000f72ef0
<4> [583.294636] PKRU: 55555554
<4> [583.294637] Call Trace:
<4> [583.294638] <TASK>
<4> [583.294642] ? lock_sync+0x100/0x100
<4> [583.294646] ? lock_release+0xd0/0x2b0
<4> [583.294652] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [583.294657] process_one_work+0x239/0x760
<4> [583.294664] worker_thread+0x200/0x3f0
<4> [583.294666] ? __pfx_worker_thread+0x10/0x10
<4> [583.294669] kthread+0x10d/0x150
<4> [583.294672] ? __pfx_kthread+0x10/0x10
<4> [583.294675] ret_from_fork+0x3d4/0x480
<4> [583.294677] ? __pfx_kthread+0x10/0x10
<4> [583.294681] ret_from_fork_asm+0x1a/0x30
<4> [583.294688] </TASK>
<4> [583.294689] irq event stamp: 1008097
<4> [583.294690] hardirqs last enabled at (1008103): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [583.294693] hardirqs last disabled at (1008108): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [583.294695] softirqs last enabled at (1006988): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.294698] softirqs last disabled at (1006979): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [583.294700] ---[ end trace 0000000000000000 ]---
<6> [583.294702] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [583.294773] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [583.294780] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [583.295188] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [583.295394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [583.295483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [583.295576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [583.295664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [583.295751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [583.295837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [583.295925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [583.296017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [583.296102] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [583.296199] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [583.296324] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [583.297331] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [583.308053] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
|