Results for igt@kms_plane_lowres@tiling-4@pipe-d-hdmi-a-3

Result: Dmesg-Warn 395 Warning(s)

i915_display_info4 igt_runner4 results4.json results4-xe-load.json guc_logs4.tar i915_display_info_post_exec4 boot4 dmesg4

DetailValue
Duration 13.00 seconds
Hostname
shard-bmg-3
Igt-Version
IGT-Version: 2.4-g2d1f7b290 (x86_64) (Linux: 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ x86_64)
Out
Starting dynamic subtest: pipe-D-HDMI-A-3
Dynamic subtest pipe-D-HDMI-A-3: SUCCESS (12.999s)
Subtest tiling-4: SUCCESS (105.713s)
Err
Starting dynamic subtest: pipe-D-HDMI-A-3
Dynamic subtest pipe-D-HDMI-A-3: SUCCESS (12.999s)
Subtest tiling-4: SUCCESS (105.713s)
Dmesg

<6> [349.119217] [IGT] kms_plane_lowres: starting dynamic subtest pipe-D-HDMI-A-3
<5> [349.119481] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278430, lrc_seqno=278430, guc_id=0, flags=0x73 in no process [-1]
<7> [349.119488] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.119578] ------------[ cut here ]------------
<4> [349.119580] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.119582] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:44/5138
<4> [349.119684] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class
<7> [349.119762] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:663]
<4> [349.119771] intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.119788] CPU: 0 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.119792] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.119794] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.119796] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.119804] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.119897] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.119899] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [349.119902] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.119904] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.119906] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.119907] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.119909] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.119910] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [349.119912] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.119914] CR2: 00007f7965941000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [349.119916] PKRU: 55555554
<4> [349.119918] Call Trace:
<4> [349.119919] <TASK>
<4> [349.119924] ? lock_sync+0x100/0x100
<4> [349.119931] ? lock_release+0xd0/0x2b0
<4> [349.119938] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.119945] process_one_work+0x239/0x760
<4> [349.119954] worker_thread+0x200/0x3f0
<4> [349.119957] ? __pfx_worker_thread+0x10/0x10
<4> [349.119960] kthread+0x10d/0x150
<4> [349.119965] ? __pfx_kthread+0x10/0x10
<4> [349.119969] ret_from_fork+0x3d4/0x480
<4> [349.119972] ? __pfx_kthread+0x10/0x10
<4> [349.119976] ret_from_fork_asm+0x1a/0x30
<4> [349.119987] </TASK>
<4> [349.119989] irq event stamp: 2022267
<4> [349.119990] hardirqs last enabled at (2022273): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.119994] hardirqs last disabled at (2022278): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.119997] softirqs last enabled at (2021434): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.120000] softirqs last disabled at (2021423): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.120003] ---[ end trace 0000000000000000 ]---
<6> [349.120006] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.120097] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.120110] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.120186] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.120465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.120620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.120739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.120843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.120959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.121098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.121212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.121340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.121472] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.121572] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.121743] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.122956] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.133499] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.133762] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.134812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.134906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.134994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.135084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.135192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.135280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.135373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.135467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.135558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.135648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.135737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.135823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.135907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.135992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.136073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.136159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.136240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.136321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.136402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.136492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.136582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.136670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.136756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.136838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.136924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.137006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.137085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.137176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.137258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.137340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.137424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.137505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.137590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.137675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.137759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.137840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.137919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.137995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.138072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.138155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.138237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.138319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.138400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.138478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.138559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.138636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.138713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.138789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.138866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.138940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.139015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.139093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.139184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.139263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.139340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.139418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.139494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.139569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.139648] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.139765] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.139769] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278430, lrc_seqno=278430, guc_id=0, flags=0x73 in no process [-1]
<7> [349.139772] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.139831] ------------[ cut here ]------------
<4> [349.139832] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.139834] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:40/5134
<4> [349.139904] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.139969] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.139974] CPU: 8 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.139977] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.139978] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.139980] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.139985] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.140055] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.140057] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [349.140059] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.140061] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.140062] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.140063] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.140064] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.140066] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.140067] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.140069] CR2: 00007f7967079000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [349.140070] PKRU: 55555554
<4> [349.140071] Call Trace:
<4> [349.140072] <TASK>
<4> [349.140077] ? lock_sync+0x100/0x100
<4> [349.140081] ? lock_release+0xd0/0x2b0
<4> [349.140087] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.140092] process_one_work+0x239/0x760
<4> [349.140099] worker_thread+0x200/0x3f0
<4> [349.140101] ? __pfx_worker_thread+0x10/0x10
<4> [349.140104] kthread+0x10d/0x150
<4> [349.140107] ? __pfx_kthread+0x10/0x10
<4> [349.140110] ret_from_fork+0x3d4/0x480
<4> [349.140112] ? __pfx_kthread+0x10/0x10
<4> [349.140116] ret_from_fork_asm+0x1a/0x30
<4> [349.140123] </TASK>
<4> [349.140124] irq event stamp: 1999723
<4> [349.140125] hardirqs last enabled at (1999729): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.140128] hardirqs last disabled at (1999734): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.140130] softirqs last enabled at (1998828): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.140138] softirqs last disabled at (1998821): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.140141] ---[ end trace 0000000000000000 ]---
<6> [349.140143] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.140213] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.140219] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.140626] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.140828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.140914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.141006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.141093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.141191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.141275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.141359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.141444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.141523] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.141617] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.141733] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.142742] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.153468] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.153717] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.154753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.154832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.154908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.154986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.155061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.155142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.155216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.155294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.155371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.155444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.155518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.155590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.155662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.155734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.155806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.155879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.155953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.156026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.156099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.156188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.156269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.156347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.156429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.156510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.156589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.156671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.156752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.156837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.156921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.157002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.157081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.157165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.157250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.157335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.157418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.157501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.157584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.157664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.157743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.157818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.157896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.157972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.158049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.158125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.158218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.158294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.158371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.158448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.158525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.158601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.158680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.158762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.158842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.158921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.158998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.159075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.159154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.159230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.159309] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.159423] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.159428] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278430, lrc_seqno=278430, guc_id=0, flags=0x73 in no process [-1]
<7> [349.159430] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.159488] ------------[ cut here ]------------
<4> [349.159489] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.159491] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:40/5134
<4> [349.159562] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.159625] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.159629] CPU: 8 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.159632] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.159633] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.159634] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.159639] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.159709] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.159711] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [349.159713] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.159714] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.159716] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.159717] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.159718] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.159719] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.159721] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.159722] CR2: 00007f7967079000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [349.159724] PKRU: 55555554
<4> [349.159725] Call Trace:
<4> [349.159726] <TASK>
<4> [349.159730] ? lock_sync+0x100/0x100
<4> [349.159734] ? lock_release+0xd0/0x2b0
<4> [349.159739] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.159745] process_one_work+0x239/0x760
<4> [349.159751] worker_thread+0x200/0x3f0
<4> [349.159754] ? __pfx_worker_thread+0x10/0x10
<4> [349.159756] kthread+0x10d/0x150
<4> [349.159759] ? __pfx_kthread+0x10/0x10
<4> [349.159762] ret_from_fork+0x3d4/0x480
<4> [349.159764] ? __pfx_kthread+0x10/0x10
<4> [349.159768] ret_from_fork_asm+0x1a/0x30
<4> [349.159775] </TASK>
<4> [349.159776] irq event stamp: 2002875
<4> [349.159777] hardirqs last enabled at (2002881): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.159780] hardirqs last disabled at (2002886): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.159782] softirqs last enabled at (2001820): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.159784] softirqs last disabled at (2001813): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.159787] ---[ end trace 0000000000000000 ]---
<7> [349.161152] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.161301] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.178494] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278446, lrc_seqno=278446, guc_id=0, flags=0x73 in no process [-1]
<7> [349.178499] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.178573] ------------[ cut here ]------------
<4> [349.178575] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.178576] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:38/5132
<4> [349.178656] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.178719] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.178724] CPU: 2 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.178727] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.178728] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.178730] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.178735] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.178811] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.178813] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.178815] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.178816] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.178818] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.178819] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.178820] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.178821] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [349.178823] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.178824] CR2: 00007f7966135000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.178826] PKRU: 55555554
<4> [349.178827] Call Trace:
<4> [349.178828] <TASK>
<4> [349.178832] ? lock_sync+0x100/0x100
<4> [349.178837] ? lock_release+0xd0/0x2b0
<4> [349.178842] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.178848] process_one_work+0x239/0x760
<4> [349.178854] worker_thread+0x200/0x3f0
<4> [349.178857] ? __pfx_worker_thread+0x10/0x10
<4> [349.178859] kthread+0x10d/0x150
<4> [349.178862] ? __pfx_kthread+0x10/0x10
<4> [349.178866] ret_from_fork+0x3d4/0x480
<4> [349.178868] ? __pfx_kthread+0x10/0x10
<4> [349.178871] ret_from_fork_asm+0x1a/0x30
<4> [349.178879] </TASK>
<4> [349.178880] irq event stamp: 1855915
<4> [349.178881] hardirqs last enabled at (1855921): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.178884] hardirqs last disabled at (1855926): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.178886] softirqs last enabled at (1854698): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [349.178889] softirqs last disabled at (1854696): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [349.178891] ---[ end trace 0000000000000000 ]---
<6> [349.178893] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.178968] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.178975] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.179396] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.179607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.179698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.179792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.179880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.179969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.180056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.180154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.180287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.180375] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.180472] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.180588] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.181617] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.192643] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.192893] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.193930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.194009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.194086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.194173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.194248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.194324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.194399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.194473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.194548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.194623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.194697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.194772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.194845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.194918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.194990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.195063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.195141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.195212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.195285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.195365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.195445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.195525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.195608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.195686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.195763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.195840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.195917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.195995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.196075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.196157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.196236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.196318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.196405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.196492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.196577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.196659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.196740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.196817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.196894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.196968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.197050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.197131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.197225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.197303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.197383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.197459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.197536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.197611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.197689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.197764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.197837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.197916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.197991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.198067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.198146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.198223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.198299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.198380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.198462] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.198577] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.198581] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278446, lrc_seqno=278446, guc_id=0, flags=0x73 in no process [-1]
<7> [349.198583] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.198641] ------------[ cut here ]------------
<4> [349.198642] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.198644] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:40/5134
<4> [349.198715] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.198778] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.198783] CPU: 8 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.198786] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.198787] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.198789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.198794] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.198863] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.198864] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [349.198867] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.198868] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.198869] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.198870] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.198872] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.198873] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.198875] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.198876] CR2: 00007f7967079000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [349.198877] PKRU: 55555554
<4> [349.198879] Call Trace:
<4> [349.198880] <TASK>
<4> [349.198884] ? lock_sync+0x100/0x100
<4> [349.198889] ? lock_release+0xd0/0x2b0
<4> [349.198894] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.198900] process_one_work+0x239/0x760
<4> [349.198906] worker_thread+0x200/0x3f0
<4> [349.198909] ? __pfx_worker_thread+0x10/0x10
<4> [349.198911] kthread+0x10d/0x150
<4> [349.198914] ? __pfx_kthread+0x10/0x10
<4> [349.198918] ret_from_fork+0x3d4/0x480
<4> [349.198920] ? __pfx_kthread+0x10/0x10
<4> [349.198923] ret_from_fork_asm+0x1a/0x30
<4> [349.198930] </TASK>
<4> [349.198931] irq event stamp: 2006271
<4> [349.198933] hardirqs last enabled at (2006277): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.198935] hardirqs last disabled at (2006282): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.198938] softirqs last enabled at (2005004): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.198940] softirqs last disabled at (2004995): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.198942] ---[ end trace 0000000000000000 ]---
<6> [349.198944] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.199013] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.199019] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.200267] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.200376] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.200482] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.200693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.200786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.200881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.200973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.201061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.201153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.201240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.201327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.201410] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.201505] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.201624] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.202633] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.211144] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling AUX_TC2
<7> [349.211294] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling DC_off
<7> [349.211411] xe 0000:03:00.0: [drm:skl_enable_dc6 [xe]] Enabling DC6
<7> [349.211511] xe 0000:03:00.0: [drm:gen9_set_dc_state.part.0 [xe]] Setting DC state from 00 to 02
<7> [349.211629] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling always-on
<7> [349.213357] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.213612] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.214624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.214704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.214782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.214859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.214936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.215014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.215092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.215181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.215255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.215328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.215402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.215475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.215549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.215622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.215694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.215766] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.215840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.215913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.215985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.216064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.216152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.216233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.216313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.216391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.216469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.216550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.216629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.216710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.216790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.216869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.216947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.217025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.217108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.217203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.217291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.217373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.217455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.217532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.217610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.217685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.217806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.217940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.218081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.218219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.218341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.218461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.218582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.218685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.218767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.218845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.218921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.219006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.219089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.219181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.219258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.219337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.219413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.219488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.219566] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.219681] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.219686] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278446, lrc_seqno=278446, guc_id=0, flags=0x73 in no process [-1]
<7> [349.219689] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.219747] ------------[ cut here ]------------
<4> [349.219748] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.219749] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:26/3887
<4> [349.219820] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.219885] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.219890] CPU: 6 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.219893] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.219894] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.219896] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.219901] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.219973] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.219975] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.219977] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.219978] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.219980] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.219981] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.219982] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.219984] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.219985] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.219987] CR2: 00007f79663b5010 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.219988] PKRU: 55555554
<4> [349.219989] Call Trace:
<4> [349.219990] <TASK>
<4> [349.219995] ? lock_sync+0x100/0x100
<4> [349.219999] ? lock_release+0xd0/0x2b0
<4> [349.220005] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.220011] process_one_work+0x239/0x760
<4> [349.220017] worker_thread+0x200/0x3f0
<4> [349.220020] ? __pfx_worker_thread+0x10/0x10
<4> [349.220022] kthread+0x10d/0x150
<4> [349.220025] ? __pfx_kthread+0x10/0x10
<4> [349.220029] ret_from_fork+0x3d4/0x480
<4> [349.220031] ? __pfx_kthread+0x10/0x10
<4> [349.220034] ret_from_fork_asm+0x1a/0x30
<4> [349.220042] </TASK>
<4> [349.220043] irq event stamp: 1407879
<4> [349.220044] hardirqs last enabled at (1407885): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.220047] hardirqs last disabled at (1407890): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.220049] softirqs last enabled at (1406750): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.220052] softirqs last disabled at (1406745): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.220054] ---[ end trace 0000000000000000 ]---
<7> [349.221348] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.221454] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.222446] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278447, lrc_seqno=278447, guc_id=0, flags=0x73 in no process [-1]
<7> [349.222449] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [349.222483] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:664]
<4> [349.222512] ------------[ cut here ]------------
<4> [349.222514] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.222515] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:26/3887
<4> [349.222585] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.222649] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.222653] CPU: 6 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.222656] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.222657] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.222658] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.222663] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.222733] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.222735] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.222737] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.222738] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.222740] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.222741] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.222742] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.222743] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.222745] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.222746] CR2: 00007f79663b5010 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.222748] PKRU: 55555554
<4> [349.222749] Call Trace:
<4> [349.222750] <TASK>
<4> [349.222753] ? lock_sync+0x100/0x100
<4> [349.222758] ? lock_release+0xd0/0x2b0
<4> [349.222763] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.222769] process_one_work+0x239/0x760
<4> [349.222775] worker_thread+0x200/0x3f0
<4> [349.222778] ? __pfx_worker_thread+0x10/0x10
<4> [349.222780] kthread+0x10d/0x150
<4> [349.222783] ? __pfx_kthread+0x10/0x10
<4> [349.222787] ret_from_fork+0x3d4/0x480
<4> [349.222789] ? __pfx_kthread+0x10/0x10
<4> [349.222792] ret_from_fork_asm+0x1a/0x30
<4> [349.222799] </TASK>
<4> [349.222800] irq event stamp: 1409001
<4> [349.222801] hardirqs last enabled at (1409007): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.222804] hardirqs last disabled at (1409012): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.222806] softirqs last enabled at (1406750): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.222809] softirqs last disabled at (1406745): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.222811] ---[ end trace 0000000000000000 ]---
<6> [349.222813] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.222884] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.222890] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.223068] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.223321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.223421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.223521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.223616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.223780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.223878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.223968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.224110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.224220] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.224333] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.224485] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.225726] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.236143] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.236396] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.237406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.237486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.237563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.237640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.237715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.237790] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.237867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.237945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.238021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.238095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.238183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.238258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.238334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.238410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.238485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.238560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.238634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.238709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.238782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.238861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.238942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.239023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.239102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.239191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.239269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.239347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.239424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.239503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.239582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.239660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.239738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.239817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.239900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.239982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.240065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.240153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.240236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.240316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.240393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.240469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.240549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.240629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.240708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.240785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.240865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.240942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.241019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.241094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.241204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.241280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.241355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.241437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.241516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.241594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.241670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.241746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.241821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.241896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.241974] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.242089] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.242094] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278447, lrc_seqno=278447, guc_id=0, flags=0x73 in no process [-1]
<7> [349.242096] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.242169] ------------[ cut here ]------------
<4> [349.242170] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.242172] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:40/5134
<4> [349.242244] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.242308] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.242313] CPU: 8 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.242316] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.242317] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.242319] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.242324] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.242394] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.242396] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [349.242398] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.242400] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.242401] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.242402] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.242403] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.242405] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.242406] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.242408] CR2: 00007f7967079000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [349.242409] PKRU: 55555554
<4> [349.242410] Call Trace:
<4> [349.242411] <TASK>
<4> [349.242415] ? lock_sync+0x100/0x100
<4> [349.242420] ? lock_release+0xd0/0x2b0
<4> [349.242425] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.242431] process_one_work+0x239/0x760
<4> [349.242437] worker_thread+0x200/0x3f0
<4> [349.242440] ? __pfx_worker_thread+0x10/0x10
<4> [349.242442] kthread+0x10d/0x150
<4> [349.242445] ? __pfx_kthread+0x10/0x10
<4> [349.242449] ret_from_fork+0x3d4/0x480
<4> [349.242451] ? __pfx_kthread+0x10/0x10
<4> [349.242454] ret_from_fork_asm+0x1a/0x30
<4> [349.242461] </TASK>
<4> [349.242463] irq event stamp: 2009871
<4> [349.242464] hardirqs last enabled at (2009877): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.242467] hardirqs last disabled at (2009882): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.242469] softirqs last enabled at (2008760): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.242471] softirqs last disabled at (2008751): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.242474] ---[ end trace 0000000000000000 ]---
<6> [349.242475] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.242546] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.242552] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.243788] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.243892] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.244001] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.244210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.244300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.244394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.244483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.244570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.244657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.244743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.244829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.244909] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.245003] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.245117] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.246139] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.256860] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.257110] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.258118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.258208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.258285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.258361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.258437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.258514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.258588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.258664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.258737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.258810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.258883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.258957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.259030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.259105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.259188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.259265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.259343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.259419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.259493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.259573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.259652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.259729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.259808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.259884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.259963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.260042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.260120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.260210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.260290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.260368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.260446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.260524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.260607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.260695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.260781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.260863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.260944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.261020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.261097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.261180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.261259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.261336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.261411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.261487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.261566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.261641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.261717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.261793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.261874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.261952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.262030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.262109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.262194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.262271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.262347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.262427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.262505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.262580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.262659] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.262774] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.262779] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278447, lrc_seqno=278447, guc_id=0, flags=0x73 in no process [-1]
<7> [349.262781] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.262839] ------------[ cut here ]------------
<4> [349.262840] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.262842] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:38/5132
<4> [349.262912] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.262975] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.262980] CPU: 2 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.262983] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.262984] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.262985] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.262990] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.263059] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.263061] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.263063] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.263065] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.263066] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.263067] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.263069] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.263070] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [349.263071] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.263073] CR2: 00007f7966135000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.263074] PKRU: 55555554
<4> [349.263076] Call Trace:
<4> [349.263077] <TASK>
<4> [349.263081] ? lock_sync+0x100/0x100
<4> [349.263085] ? lock_release+0xd0/0x2b0
<4> [349.263090] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.263096] process_one_work+0x239/0x760
<4> [349.263102] worker_thread+0x200/0x3f0
<4> [349.263105] ? __pfx_worker_thread+0x10/0x10
<4> [349.263107] kthread+0x10d/0x150
<4> [349.263110] ? __pfx_kthread+0x10/0x10
<4> [349.263114] ret_from_fork+0x3d4/0x480
<4> [349.263116] ? __pfx_kthread+0x10/0x10
<4> [349.263119] ret_from_fork_asm+0x1a/0x30
<4> [349.263127] </TASK>
<4> [349.263128] irq event stamp: 1859437
<4> [349.263129] hardirqs last enabled at (1859443): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.263132] hardirqs last disabled at (1859448): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.263138] softirqs last enabled at (1858542): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.263140] softirqs last disabled at (1858537): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.263143] ---[ end trace 0000000000000000 ]---
<7> [349.264173] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.264280] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.271548] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278450, lrc_seqno=278450, guc_id=0, flags=0x73 in no process [-1]
<7> [349.271553] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.271620] ------------[ cut here ]------------
<4> [349.271621] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.271623] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:40/5134
<4> [349.271698] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.271762] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.271767] CPU: 0 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.271770] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.271771] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.271772] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.271778] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.271849] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.271851] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [349.271853] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.271855] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.271856] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.271857] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.271858] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.271860] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [349.271861] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.271863] CR2: 00007f7965941000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [349.271864] PKRU: 55555554
<4> [349.271865] Call Trace:
<4> [349.271867] <TASK>
<4> [349.271870] ? lock_sync+0x100/0x100
<4> [349.271875] ? lock_release+0xd0/0x2b0
<4> [349.271881] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.271886] process_one_work+0x239/0x760
<4> [349.271893] worker_thread+0x200/0x3f0
<4> [349.271895] ? __pfx_worker_thread+0x10/0x10
<4> [349.271898] kthread+0x10d/0x150
<4> [349.271901] ? __pfx_kthread+0x10/0x10
<4> [349.271904] ret_from_fork+0x3d4/0x480
<4> [349.271906] ? __pfx_kthread+0x10/0x10
<4> [349.271910] ret_from_fork_asm+0x1a/0x30
<4> [349.271917] </TASK>
<4> [349.271918] irq event stamp: 2011115
<4> [349.271919] hardirqs last enabled at (2011121): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.271922] hardirqs last disabled at (2011126): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.271924] softirqs last enabled at (2010220): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [349.271927] softirqs last disabled at (2010218): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [349.271929] ---[ end trace 0000000000000000 ]---
<6> [349.271931] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.272004] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.272010] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.272184] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.272391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.272480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.272573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.272660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.272746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.272830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.272915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.273000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.273082] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.273190] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.273307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.274317] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.284142] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.284392] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.285414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.285492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.285568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.285646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.285724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.285799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.285874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.285948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.286023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.286096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.286179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.286253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.286329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.286405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.286481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.286556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.286630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.286703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.286777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.286857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.286939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.287020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.287100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.287189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.287266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.287342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.287417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.287495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.287574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.287650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.287728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.287806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.287888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.287972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.288056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.288144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.288226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.288305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.288384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.288461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.288540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.288619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.288696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.288772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.288852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.288931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.289008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.289087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.289173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.289250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.289325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.289404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.289483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.289563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.289639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.289717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.289793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.289868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.289946] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.290063] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.290067] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278450, lrc_seqno=278450, guc_id=0, flags=0x73 in no process [-1]
<7> [349.290069] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.290128] ------------[ cut here ]------------
<4> [349.290129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.290130] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:44/5138
<4> [349.290216] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.290280] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.290285] CPU: 2 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.290288] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.290289] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.290290] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.290295] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.290364] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.290366] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [349.290368] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.290370] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.290371] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.290372] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.290373] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.290375] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [349.290376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.290378] CR2: 00007f7966135000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.290379] PKRU: 55555554
<4> [349.290380] Call Trace:
<4> [349.290381] <TASK>
<4> [349.290385] ? lock_sync+0x100/0x100
<4> [349.290390] ? lock_release+0xd0/0x2b0
<4> [349.290395] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.290401] process_one_work+0x239/0x760
<4> [349.290407] worker_thread+0x200/0x3f0
<4> [349.290410] ? __pfx_worker_thread+0x10/0x10
<4> [349.290412] kthread+0x10d/0x150
<4> [349.290415] ? __pfx_kthread+0x10/0x10
<4> [349.290419] ret_from_fork+0x3d4/0x480
<4> [349.290421] ? __pfx_kthread+0x10/0x10
<4> [349.290424] ret_from_fork_asm+0x1a/0x30
<4> [349.290431] </TASK>
<4> [349.290432] irq event stamp: 2026517
<4> [349.290433] hardirqs last enabled at (2026523): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.290436] hardirqs last disabled at (2026528): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.290438] softirqs last enabled at (2025500): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.290441] softirqs last disabled at (2025493): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.290443] ---[ end trace 0000000000000000 ]---
<6> [349.290445] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.290515] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.290521] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.291756] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.291859] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.291965] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.292180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.292273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.292366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.292453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.292539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.292625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.292763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.292912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.293056] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.293231] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.293392] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.294484] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.305482] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.305732] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.306801] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.306926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.307027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.307107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.307199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.307278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.307355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.307430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.307503] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.307577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.307650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.307723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.307797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.307869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.307942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.308015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.308087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.308168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.308241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.308322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.308404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.308483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.308566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.308645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.308723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.308800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.308877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.308956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.309036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.309114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.309204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.309281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.309364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.309447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.309531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.309615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.309696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.309774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.309852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.309927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.310005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.310083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.310170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.310249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.310332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.310409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.310487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.310563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.310642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.310717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.310793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.310871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.310946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.311023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.311102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.311191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.311267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.311342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.311421] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.311538] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.311542] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278450, lrc_seqno=278450, guc_id=0, flags=0x73 in no process [-1]
<7> [349.311545] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.311603] ------------[ cut here ]------------
<4> [349.311605] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.311606] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [349.311678] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.311742] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.311746] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.311749] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.311750] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.311752] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.311757] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.311828] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.311830] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.311832] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.311833] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.311835] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.311836] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.311837] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.311839] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.311840] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.311842] CR2: 00007f79663b5010 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.311843] PKRU: 55555554
<4> [349.311844] Call Trace:
<4> [349.311845] <TASK>
<4> [349.311850] ? lock_sync+0x100/0x100
<4> [349.311854] ? lock_release+0xd0/0x2b0
<4> [349.311860] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.311866] process_one_work+0x239/0x760
<4> [349.311872] worker_thread+0x200/0x3f0
<4> [349.311875] ? __pfx_worker_thread+0x10/0x10
<4> [349.311877] kthread+0x10d/0x150
<4> [349.311880] ? __pfx_kthread+0x10/0x10
<4> [349.311884] ret_from_fork+0x3d4/0x480
<4> [349.311886] ? __pfx_kthread+0x10/0x10
<4> [349.311889] ret_from_fork_asm+0x1a/0x30
<4> [349.311896] </TASK>
<4> [349.311897] irq event stamp: 1862733
<4> [349.311899] hardirqs last enabled at (1862739): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.311901] hardirqs last disabled at (1862744): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.311904] softirqs last enabled at (1861474): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.311906] softirqs last disabled at (1861465): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.311908] ---[ end trace 0000000000000000 ]---
<7> [349.312943] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.313053] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.313778] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278451, lrc_seqno=278451, guc_id=0, flags=0x73 in no process [-1]
<7> [349.313781] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [349.313814] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:665]
<4> [349.313843] ------------[ cut here ]------------
<4> [349.313844] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.313845] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [349.313917] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.313980] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.313984] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.313987] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.313988] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.313989] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.313994] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.314064] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.314066] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.314068] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.314069] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.314071] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.314072] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.314073] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.314074] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.314076] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.314077] CR2: 00007f79663b5010 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.314078] PKRU: 55555554
<4> [349.314080] Call Trace:
<4> [349.314081] <TASK>
<4> [349.314085] ? lock_sync+0x100/0x100
<4> [349.314089] ? lock_release+0xd0/0x2b0
<4> [349.314094] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.314100] process_one_work+0x239/0x760
<4> [349.314106] worker_thread+0x200/0x3f0
<4> [349.314108] ? __pfx_worker_thread+0x10/0x10
<4> [349.314111] kthread+0x10d/0x150
<4> [349.314113] ? __pfx_kthread+0x10/0x10
<4> [349.314117] ret_from_fork+0x3d4/0x480
<4> [349.314119] ? __pfx_kthread+0x10/0x10
<4> [349.314122] ret_from_fork_asm+0x1a/0x30
<4> [349.314130] </TASK>
<4> [349.314131] irq event stamp: 1863691
<4> [349.314132] hardirqs last enabled at (1863697): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.314140] hardirqs last disabled at (1863710): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.314143] softirqs last enabled at (1863706): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.314145] softirqs last disabled at (1863701): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.314148] ---[ end trace 0000000000000000 ]---
<6> [349.314149] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.314221] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.314227] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.314282] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.314543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.314652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.314747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.314835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.314919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.315002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.315087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.315268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.315353] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.315463] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.315581] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.316712] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.327143] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.327395] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.328413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.328506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.328584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.328662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.328738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.328815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.328891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.328967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.329041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.329115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.329203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.329286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.329359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.329431] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.329504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.329577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.329649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.329721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.329793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.329873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.329953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.330033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.330114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.330205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.330293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.330369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.330445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.330525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.330605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.330682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.330760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.330838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.330922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.331005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.331089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.331227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.331307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.331383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.331462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.331541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.331621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.331699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.331777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.331853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.331933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.332011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.332092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.332188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.332279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.332359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.332436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.332514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.332590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.332666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.332743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.332819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.332892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.332970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.333054] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.333177] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.333184] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278451, lrc_seqno=278451, guc_id=0, flags=0x73 in no process [-1]
<7> [349.333188] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.333254] ------------[ cut here ]------------
<4> [349.333255] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.333257] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:40/5134
<4> [349.333328] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.333392] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.333397] CPU: 0 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.333400] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.333401] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.333402] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.333407] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.333476] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.333477] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [349.333480] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.333481] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.333482] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.333484] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.333485] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.333486] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [349.333488] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.333489] CR2: 00007f7965941000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [349.333491] PKRU: 55555554
<4> [349.333492] Call Trace:
<4> [349.333493] <TASK>
<4> [349.333497] ? lock_sync+0x100/0x100
<4> [349.333501] ? lock_release+0xd0/0x2b0
<4> [349.333507] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.333512] process_one_work+0x239/0x760
<4> [349.333519] worker_thread+0x200/0x3f0
<4> [349.333521] ? __pfx_worker_thread+0x10/0x10
<4> [349.333524] kthread+0x10d/0x150
<4> [349.333526] ? __pfx_kthread+0x10/0x10
<4> [349.333530] ret_from_fork+0x3d4/0x480
<4> [349.333532] ? __pfx_kthread+0x10/0x10
<4> [349.333535] ret_from_fork_asm+0x1a/0x30
<4> [349.333543] </TASK>
<4> [349.333544] irq event stamp: 2014879
<4> [349.333545] hardirqs last enabled at (2014885): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.333548] hardirqs last disabled at (2014890): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.333550] softirqs last enabled at (2013950): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.333553] softirqs last disabled at (2013943): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.333555] ---[ end trace 0000000000000000 ]---
<6> [349.333557] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.333626] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.333632] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.334871] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.334977] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.335082] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.335311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.335410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.335509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.335602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.335693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.335783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.335874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.335965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.336055] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.336168] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.336291] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.337304] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.347145] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.347402] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.348412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.348497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.348577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.348662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.348746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.348827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.348906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.348986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.349064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.349144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.349224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.349305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.349386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.349464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.349543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.349620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.349698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.349774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.349852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.349938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.350022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.350105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.350200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.350289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.350371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.350452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.350533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.350615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.350697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.350780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.350862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.350944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.351033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.351119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.351217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.351303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.351388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.351470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.351551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.351630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.351714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.351797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.351879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.351965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.352054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.352141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.352223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.352304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.352386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.352466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.352547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.352631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.352713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.352795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.352875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.352957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.353037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.353116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.353209] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.353328] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.353332] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278451, lrc_seqno=278451, guc_id=0, flags=0x73 in no process [-1]
<7> [349.353335] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.353397] ------------[ cut here ]------------
<4> [349.353398] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.353399] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [349.353477] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.353541] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.353546] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.353549] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.353550] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.353551] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.353557] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.353633] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.353635] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.353637] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.353639] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.353640] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.353641] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.353643] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.353644] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.353646] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.353647] CR2: 00007f79671c7000 CR3: 000000011fc71001 CR4: 0000000000f72ef0
<4> [349.353648] PKRU: 55555554
<4> [349.353650] Call Trace:
<4> [349.353651] <TASK>
<4> [349.353655] ? lock_sync+0x100/0x100
<4> [349.353659] ? lock_release+0xd0/0x2b0
<4> [349.353665] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.353671] process_one_work+0x239/0x760
<4> [349.353677] worker_thread+0x200/0x3f0
<4> [349.353680] ? __pfx_worker_thread+0x10/0x10
<4> [349.353682] kthread+0x10d/0x150
<4> [349.353685] ? __pfx_kthread+0x10/0x10
<4> [349.353689] ret_from_fork+0x3d4/0x480
<4> [349.353691] ? __pfx_kthread+0x10/0x10
<4> [349.353694] ret_from_fork_asm+0x1a/0x30
<4> [349.353702] </TASK>
<4> [349.353703] irq event stamp: 1412631
<4> [349.353704] hardirqs last enabled at (1412637): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.353707] hardirqs last disabled at (1412642): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.353709] softirqs last enabled at (1411514): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.353712] softirqs last disabled at (1411509): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.353714] ---[ end trace 0000000000000000 ]---
<7> [349.354758] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.354873] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.361808] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278454, lrc_seqno=278454, guc_id=0, flags=0x73 in no process [-1]
<7> [349.361811] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.361882] ------------[ cut here ]------------
<4> [349.361884] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.361885] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [349.361961] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.362024] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.362029] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.362032] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.362033] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.362034] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.362039] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.362114] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.362115] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.362117] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.362119] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.362120] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.362121] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.362122] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.362124] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.362125] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.362126] CR2: 00007f79671c7000 CR3: 000000011fc71001 CR4: 0000000000f72ef0
<4> [349.362128] PKRU: 55555554
<4> [349.362129] Call Trace:
<4> [349.362130] <TASK>
<4> [349.362140] ? lock_sync+0x100/0x100
<4> [349.362145] ? lock_release+0xd0/0x2b0
<4> [349.362151] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.362157] process_one_work+0x239/0x760
<4> [349.362163] worker_thread+0x200/0x3f0
<4> [349.362166] ? __pfx_worker_thread+0x10/0x10
<4> [349.362168] kthread+0x10d/0x150
<4> [349.362172] ? __pfx_kthread+0x10/0x10
<4> [349.362175] ret_from_fork+0x3d4/0x480
<4> [349.362177] ? __pfx_kthread+0x10/0x10
<4> [349.362181] ret_from_fork_asm+0x1a/0x30
<4> [349.362189] </TASK>
<4> [349.362190] irq event stamp: 1413571
<4> [349.362191] hardirqs last enabled at (1413577): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.362194] hardirqs last disabled at (1413582): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.362197] softirqs last enabled at (1413498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.362199] softirqs last disabled at (1413487): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.362202] ---[ end trace 0000000000000000 ]---
<6> [349.362204] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.362279] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.362286] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.362458] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.362668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.362759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.362852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.362939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.363028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.363114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.363213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.363305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.363389] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.363485] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.363601] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.364611] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.375143] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.375393] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.376437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.376516] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.376593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.376669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.376744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.376819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.376893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.376968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.377042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.377119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.377211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.377287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.377362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.377435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.377509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.377583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.377656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.377729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.377803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.377883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.377966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.378047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.378126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.378215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.378291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.378368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.378442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.378519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.378599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.378676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.378756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.378835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.378917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.378999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.379081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.379166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.379245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.379321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.379399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.379474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.379556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.379637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.379715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.379790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.379868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.379944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.380019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.380095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.380186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.380262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.380336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.380414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.380491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.380567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.380642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.380721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.380799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.380877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.380956] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.381073] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.381077] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278454, lrc_seqno=278454, guc_id=0, flags=0x73 in no process [-1]
<7> [349.381080] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.381145] ------------[ cut here ]------------
<4> [349.381146] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.381148] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:44/5138
<4> [349.381219] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.381282] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.381287] CPU: 2 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.381290] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.381291] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.381292] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.381297] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.381366] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.381368] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [349.381370] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.381372] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.381373] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.381374] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.381375] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.381377] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [349.381378] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.381380] CR2: 00007f7966135000 CR3: 0000000124481006 CR4: 0000000000f72ef0
<4> [349.381381] PKRU: 55555554
<4> [349.381382] Call Trace:
<4> [349.381383] <TASK>
<4> [349.381387] ? lock_sync+0x100/0x100
<4> [349.381392] ? lock_release+0xd0/0x2b0
<4> [349.381397] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.381403] process_one_work+0x239/0x760
<4> [349.381409] worker_thread+0x200/0x3f0
<4> [349.381411] ? __pfx_worker_thread+0x10/0x10
<4> [349.381414] kthread+0x10d/0x150
<4> [349.381417] ? __pfx_kthread+0x10/0x10
<4> [349.381420] ret_from_fork+0x3d4/0x480
<4> [349.381422] ? __pfx_kthread+0x10/0x10
<4> [349.381426] ret_from_fork_asm+0x1a/0x30
<4> [349.381433] </TASK>
<4> [349.381434] irq event stamp: 2030537
<4> [349.381435] hardirqs last enabled at (2030543): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.381438] hardirqs last disabled at (2030548): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.381440] softirqs last enabled at (2029614): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.381442] softirqs last disabled at (2029605): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.381445] ---[ end trace 0000000000000000 ]---
<6> [349.381447] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.381515] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.381522] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.382765] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.382868] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.382973] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.383190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.383282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.383377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.383465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.383552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.383637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.383722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.383808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.383890] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.383984] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.384100] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.385117] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.395860] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.396114] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.397133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.397230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.397305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.397382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.397458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.397533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.397610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.397685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.397759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.397833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.397908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.397981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.398055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.398130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.398223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.398300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.398374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.398449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.398523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.398604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.398685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.398765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.398845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.398924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.399001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.399077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.399163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.399245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.399326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.399404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.399482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.399561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.399645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.399729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.399814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.399894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.399979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.400059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.400144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.400222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.400303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.400381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.400458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.400534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.400614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.400691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.400768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.400846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.400926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.401002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.401078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.401162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.401239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.401316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.401392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.401468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.401542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.401617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.401695] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.401811] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.401815] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278454, lrc_seqno=278454, guc_id=0, flags=0x73 in no process [-1]
<7> [349.401818] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.401876] ------------[ cut here ]------------
<4> [349.401877] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.401878] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [349.401951] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.402014] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.402019] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.402022] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.402023] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.402025] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.402030] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.402101] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.402103] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.402105] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.402107] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.402108] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.402109] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.402110] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.402112] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.402113] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.402115] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.402116] PKRU: 55555554
<4> [349.402117] Call Trace:
<4> [349.402119] <TASK>
<4> [349.402122] ? lock_sync+0x100/0x100
<4> [349.402127] ? lock_release+0xd0/0x2b0
<4> [349.402133] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.402143] process_one_work+0x239/0x760
<4> [349.402150] worker_thread+0x200/0x3f0
<4> [349.402153] ? __pfx_worker_thread+0x10/0x10
<4> [349.402155] kthread+0x10d/0x150
<4> [349.402158] ? __pfx_kthread+0x10/0x10
<4> [349.402162] ret_from_fork+0x3d4/0x480
<4> [349.402164] ? __pfx_kthread+0x10/0x10
<4> [349.402168] ret_from_fork_asm+0x1a/0x30
<4> [349.402175] </TASK>
<4> [349.402176] irq event stamp: 1867291
<4> [349.402177] hardirqs last enabled at (1867297): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.402180] hardirqs last disabled at (1867302): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.402182] softirqs last enabled at (1866010): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.402185] softirqs last disabled at (1866001): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.402187] ---[ end trace 0000000000000000 ]---
<7> [349.403869] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.404022] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.404467] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278455, lrc_seqno=278455, guc_id=0, flags=0x73 in no process [-1]
<7> [349.404470] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.404536] ------------[ cut here ]------------
<4> [349.404537] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.404538] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [349.404612] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei
<7> [349.404664] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:666]
<4> [349.404667] intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.404684] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.404686] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.404688] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.404689] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.404694] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.404762] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.404764] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.404766] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.404767] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.404768] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.404769] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.404770] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.404772] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.404773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.404775] CR2: 00007f79671c7000 CR3: 000000011fc71001 CR4: 0000000000f72ef0
<4> [349.404776] PKRU: 55555554
<4> [349.404777] Call Trace:
<4> [349.404778] <TASK>
<4> [349.404782] ? lock_sync+0x100/0x100
<4> [349.404786] ? lock_release+0xd0/0x2b0
<4> [349.404791] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.404797] process_one_work+0x239/0x760
<4> [349.404803] worker_thread+0x200/0x3f0
<4> [349.404806] ? __pfx_worker_thread+0x10/0x10
<4> [349.404808] kthread+0x10d/0x150
<4> [349.404811] ? __pfx_kthread+0x10/0x10
<4> [349.404815] ret_from_fork+0x3d4/0x480
<4> [349.404817] ? __pfx_kthread+0x10/0x10
<4> [349.404820] ret_from_fork_asm+0x1a/0x30
<4> [349.404827] </TASK>
<4> [349.404828] irq event stamp: 1415099
<4> [349.404829] hardirqs last enabled at (1415105): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.404832] hardirqs last disabled at (1415110): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.404834] softirqs last enabled at (1414190): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [349.404837] softirqs last disabled at (1414188): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [349.404839] ---[ end trace 0000000000000000 ]---
<6> [349.404841] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.404909] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.404915] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.404962] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.405196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.405285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.405377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.405467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.405555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.405641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.405727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.405812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.405894] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.405989] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.406106] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.407291] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.417143] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.417393] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.418393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.418472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.418548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.418625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.418700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.418776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.418851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.418926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.419000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.419077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.419156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.419232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.419307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.419381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.419456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.419530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.419603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.419675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.419747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.419828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.419910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.419991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.420072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.420180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.420256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.420332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.420408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.420487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.420565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.420644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.420722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.420805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.420893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.420979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.421063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.421148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.421227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.421304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.421380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.421455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.421534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.421611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.421686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.421761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.421840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.421915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.421991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.422067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.422146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.422225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.422303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.422383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.422460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.422538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.422613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.422689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.422762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.422838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.422918] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.423033] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.423037] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278455, lrc_seqno=278455, guc_id=0, flags=0x73 in no process [-1]
<7> [349.423039] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.423097] ------------[ cut here ]------------
<4> [349.423098] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.423100] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [349.423180] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.423245] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.423249] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.423252] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.423253] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.423254] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.423259] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.423329] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.423331] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.423333] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.423334] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.423336] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.423337] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.423338] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.423339] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [349.423341] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.423342] CR2: 00007f79671c7000 CR3: 000000011fc71001 CR4: 0000000000f72ef0
<4> [349.423344] PKRU: 55555554
<4> [349.423345] Call Trace:
<4> [349.423346] <TASK>
<4> [349.423350] ? lock_sync+0x100/0x100
<4> [349.423354] ? lock_release+0xd0/0x2b0
<4> [349.423360] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.423365] process_one_work+0x239/0x760
<4> [349.423371] worker_thread+0x200/0x3f0
<4> [349.423374] ? __pfx_worker_thread+0x10/0x10
<4> [349.423376] kthread+0x10d/0x150
<4> [349.423379] ? __pfx_kthread+0x10/0x10
<4> [349.423383] ret_from_fork+0x3d4/0x480
<4> [349.423385] ? __pfx_kthread+0x10/0x10
<4> [349.423388] ret_from_fork_asm+0x1a/0x30
<4> [349.423396] </TASK>
<4> [349.423397] irq event stamp: 1418337
<4> [349.423398] hardirqs last enabled at (1418343): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.423401] hardirqs last disabled at (1418348): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.423403] softirqs last enabled at (1417320): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.423405] softirqs last disabled at (1417313): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.423408] ---[ end trace 0000000000000000 ]---
<6> [349.424167] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.424369] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.424378] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.424433] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.424537] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.424644] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.424851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.424942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.425036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.425123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.425226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.425313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.425399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.425488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.425570] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.425664] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.425780] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.426790] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.437493] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.437744] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.438753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.438831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.438905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.438981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.439057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.439133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.439231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.439310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.439385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.439460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.439533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.439608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.439683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.439757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.439830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.439904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.439976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.440051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.440123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.440216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.440298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.440379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.440460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.440537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.440612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.440687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.440761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.440840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.440919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.440997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.441075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.441157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.441241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.441323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.441406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.441485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.441567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.441646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.441725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.441801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.441882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.441959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.442038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.442119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.442213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.442291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.442368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.442443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.442522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.442598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.442677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.442757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.442835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.442914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.442991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.443069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.443148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.443225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.443303] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.443419] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.443423] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278455, lrc_seqno=278455, guc_id=0, flags=0x73 in no process [-1]
<7> [349.443426] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.443484] ------------[ cut here ]------------
<4> [349.443486] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.443487] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:26/3887
<4> [349.443558] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.443622] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.443627] CPU: 9 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.443630] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.443631] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.443632] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.443638] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.443708] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.443710] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.443712] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.443713] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.443714] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.443716] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.443717] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.443718] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [349.443720] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.443721] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [349.443723] PKRU: 55555554
<4> [349.443724] Call Trace:
<4> [349.443725] <TASK>
<4> [349.443729] ? lock_sync+0x100/0x100
<4> [349.443734] ? lock_release+0xd0/0x2b0
<4> [349.443739] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.443745] process_one_work+0x239/0x760
<4> [349.443751] worker_thread+0x200/0x3f0
<4> [349.443754] ? __pfx_worker_thread+0x10/0x10
<4> [349.443756] kthread+0x10d/0x150
<4> [349.443759] ? __pfx_kthread+0x10/0x10
<4> [349.443763] ret_from_fork+0x3d4/0x480
<4> [349.443765] ? __pfx_kthread+0x10/0x10
<4> [349.443768] ret_from_fork_asm+0x1a/0x30
<4> [349.443775] </TASK>
<4> [349.443776] irq event stamp: 1421461
<4> [349.443777] hardirqs last enabled at (1421467): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.443780] hardirqs last disabled at (1421472): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.443783] softirqs last enabled at (1420218): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.443785] softirqs last disabled at (1420209): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.443787] ---[ end trace 0000000000000000 ]---
<7> [349.444879] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.444987] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [349.445807] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278471, lrc_seqno=278471, guc_id=0, flags=0x73 in no process [-1]
<7> [349.445811] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.445874] ------------[ cut here ]------------
<4> [349.445875] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.445877] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [349.445952] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.446015] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.446020] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.446023] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.446024] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.446025] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.446030] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.446102] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.446104] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.446106] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.446108] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.446109] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.446110] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.446111] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.446113] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.446114] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.446116] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.446117] PKRU: 55555554
<4> [349.446118] Call Trace:
<4> [349.446119] <TASK>
<4> [349.446123] ? lock_sync+0x100/0x100
<4> [349.446128] ? lock_release+0xd0/0x2b0
<4> [349.446134] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.446147] process_one_work+0x239/0x760
<4> [349.446154] worker_thread+0x200/0x3f0
<4> [349.446156] ? __pfx_worker_thread+0x10/0x10
<4> [349.446159] kthread+0x10d/0x150
<4> [349.446162] ? __pfx_kthread+0x10/0x10
<4> [349.446165] ret_from_fork+0x3d4/0x480
<4> [349.446167] ? __pfx_kthread+0x10/0x10
<4> [349.446171] ret_from_fork_asm+0x1a/0x30
<4> [349.446178] </TASK>
<4> [349.446179] irq event stamp: 1869461
<4> [349.446180] hardirqs last enabled at (1869467): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.446183] hardirqs last disabled at (1869472): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.446186] softirqs last enabled at (1869402): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.446188] softirqs last disabled at (1869397): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.446191] ---[ end trace 0000000000000000 ]---
<6> [349.446193] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.446264] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.446270] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.446358] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.446561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.446649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.446794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.446884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.446973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.447069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.447161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.447248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.447328] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.447423] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.447539] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.448665] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.459142] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.459394] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.460422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.460502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.460579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.460656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.460732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.460807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.460882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.460957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.461032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.461107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.461193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.461271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.461346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.461420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.461494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.461567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.461640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.461713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.461785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.461865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.461944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.462023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.462102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.462186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.462264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.462341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.462419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.462500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.462581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.462663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.462745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.462826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.462912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.462995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.463079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.463164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.463248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.463328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.463407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.463483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.463562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.463639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.463717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.463794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.463873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.463949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.464026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.464099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.464187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.464267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.464345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.464427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.464505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.464584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.464661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.464739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.464815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.464892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.464973] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.465090] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.465094] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278471, lrc_seqno=278471, guc_id=0, flags=0x73 in no process [-1]
<7> [349.465097] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.465165] ------------[ cut here ]------------
<4> [349.465166] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.465168] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:26/3887
<4> [349.465239] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.465302] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.465306] CPU: 9 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.465309] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.465310] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.465312] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.465317] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.465387] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.465389] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [349.465391] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.465393] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.465394] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.465395] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.465396] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.465397] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [349.465399] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.465400] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [349.465402] PKRU: 55555554
<4> [349.465403] Call Trace:
<4> [349.465404] <TASK>
<4> [349.465408] ? lock_sync+0x100/0x100
<4> [349.465412] ? lock_release+0xd0/0x2b0
<4> [349.465418] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.465423] process_one_work+0x239/0x760
<4> [349.465429] worker_thread+0x200/0x3f0
<4> [349.465432] ? __pfx_worker_thread+0x10/0x10
<4> [349.465434] kthread+0x10d/0x150
<4> [349.465437] ? __pfx_kthread+0x10/0x10
<4> [349.465441] ret_from_fork+0x3d4/0x480
<4> [349.465443] ? __pfx_kthread+0x10/0x10
<4> [349.465446] ret_from_fork_asm+0x1a/0x30
<4> [349.465453] </TASK>
<4> [349.465454] irq event stamp: 1424915
<4> [349.465455] hardirqs last enabled at (1424921): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.465458] hardirqs last disabled at (1424926): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.465460] softirqs last enabled at (1423804): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.465463] softirqs last disabled at (1423797): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.465465] ---[ end trace 0000000000000000 ]---
<6> [349.465467] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [349.465538] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [349.465544] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [349.466777] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.466882] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [349.466988] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [349.467204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [349.467296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [349.467392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [349.467483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [349.467574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [349.467660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [349.467746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [349.467832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [349.467913] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [349.468008] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [349.468123] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [349.469147] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [349.479147] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [349.479405] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [349.480430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [349.480510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [349.480586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [349.480663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [349.480742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [349.480818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [349.480892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [349.480967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [349.481040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [349.481113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [349.481197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [349.481271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [349.481344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [349.481416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [349.481490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [349.481561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [349.481635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [349.481707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [349.481778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [349.481859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [349.481940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [349.482019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [349.482099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [349.482188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [349.482269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [349.482348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [349.482426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [349.482507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [349.482586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [349.482664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [349.482743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [349.482823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [349.482909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [349.482992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [349.483075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [349.483159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [349.483238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [349.483313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [349.483392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [349.483471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [349.483551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [349.483629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [349.483706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [349.483783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [349.483863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [349.483940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [349.484017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [349.484093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [349.484215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [349.484344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [349.484458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [349.484600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [349.484715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [349.484833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [349.484949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [349.485072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [349.485207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [349.485291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [349.485374] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [349.485491] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [349.485495] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278471, lrc_seqno=278471, guc_id=0, flags=0x73 in no process [-1]
<7> [349.485498] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [349.485556] ------------[ cut here ]------------
<4> [349.485557] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [349.485559] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [349.485631] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [349.485695] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [349.485700] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [349.485702] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [349.485704] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [349.485705] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [349.485710] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [349.485780] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [349.485782] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [349.485784] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [349.485786] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [349.485787] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [349.485788] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [349.485789] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [349.485791] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [349.485792] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [349.485794] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [349.485795] PKRU: 55555554
<4> [349.485796] Call Trace:
<4> [349.485797] <TASK>
<4> [349.485802] ? lock_sync+0x100/0x100
<4> [349.485806] ? lock_release+0xd0/0x2b0
<4> [349.485811] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [349.485817] process_one_work+0x239/0x760
<4> [349.485823] worker_thread+0x200/0x3f0
<4> [349.485826] ? __pfx_worker_thread+0x10/0x10
<4> [349.485828] kthread+0x10d/0x150
<4> [349.485831] ? __pfx_kthread+0x10/0x10
<4> [349.485835] ret_from_fork+0x3d4/0x480
<4> [349.485837] ? __pfx_kthread+0x10/0x10
<4> [349.485840] ret_from_fork_asm+0x1a/0x30
<4> [349.485847] </TASK>
<4> [349.485849] irq event stamp: 1873139
<4> [349.485850] hardirqs last enabled at (1873145): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [349.485853] hardirqs last disabled at (1873150): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [349.485855] softirqs last enabled at (1872270): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.485857] softirqs last disabled at (1872247): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [349.485860] ---[ end trace 0000000000000000 ]---
<7> [349.486977] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [349.487084] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.079788] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278479, lrc_seqno=278479, guc_id=0, flags=0x73 in no process [-1]
<7> [350.079794] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.079903] ------------[ cut here ]------------
<4> [350.079905] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.079906] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:40/5134
<4> [350.079998] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.080061] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.080065] CPU: 8 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.080068] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.080069] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.080071] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.080076] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.080155] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.080157] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.080159] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.080161] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.080162] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.080163] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.080164] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.080165] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [350.080167] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.080168] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [350.080169] PKRU: 55555554
<4> [350.080170] Call Trace:
<4> [350.080171] <TASK>
<4> [350.080175] ? lock_sync+0x100/0x100
<4> [350.080180] ? lock_release+0xd0/0x2b0
<4> [350.080185] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.080190] process_one_work+0x239/0x760
<4> [350.080197] worker_thread+0x200/0x3f0
<4> [350.080200] ? __pfx_worker_thread+0x10/0x10
<4> [350.080202] kthread+0x10d/0x150
<4> [350.080205] ? __pfx_kthread+0x10/0x10
<4> [350.080209] ret_from_fork+0x3d4/0x480
<4> [350.080211] ? __pfx_kthread+0x10/0x10
<4> [350.080214] ret_from_fork_asm+0x1a/0x30
<4> [350.080222] </TASK>
<4> [350.080224] irq event stamp: 2057675
<4> [350.080225] hardirqs last enabled at (2057681): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.080228] hardirqs last disabled at (2057686): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.080230] softirqs last enabled at (2057514): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.080233] softirqs last disabled at (2057505): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.080235] ---[ end trace 0000000000000000 ]---
<6> [350.080238] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.080329] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.080337] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.080742] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.080944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.081031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.081122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.081246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.081356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.081447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.081554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.081639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.081718] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.081811] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.081925] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.083108] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.093888] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.094172] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.095230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.095314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.095388] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.095462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.095536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.095609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.095681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.095754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.095826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.095899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.095973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.096046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.096119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.096207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.096293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.096375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.096454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.096530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.096607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.096692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.096777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.096860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.096949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.097036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.097122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.097221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.097306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.097393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.097482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.097569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.097656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.097748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.097850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.097948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.098048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.098147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.098242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.098334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.098430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.098528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.098628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.098726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.098820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.098914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.099016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.099114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.099230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.099331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.099432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.099533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.099639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.099747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.099852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.099960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.100068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.100187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.100293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.100399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.100520] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.100671] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.100681] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278479, lrc_seqno=278479, guc_id=0, flags=0x73 in no process [-1]
<7> [350.100684] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.100770] ------------[ cut here ]------------
<4> [350.100771] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.100773] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:40/5134
<4> [350.100878] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.100969] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.100976] CPU: 8 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.100980] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.100982] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.100984] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.100991] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.101094] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.101097] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.101100] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.101103] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.101105] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.101106] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.101108] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.101110] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [350.101112] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.101114] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [350.101117] PKRU: 55555554
<4> [350.101118] Call Trace:
<4> [350.101120] <TASK>
<4> [350.101126] ? lock_sync+0x100/0x100
<4> [350.101133] ? lock_release+0xd0/0x2b0
<4> [350.101149] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.101157] process_one_work+0x239/0x760
<4> [350.101167] worker_thread+0x200/0x3f0
<4> [350.101172] ? __pfx_worker_thread+0x10/0x10
<4> [350.101175] kthread+0x10d/0x150
<4> [350.101180] ? __pfx_kthread+0x10/0x10
<4> [350.101185] ret_from_fork+0x3d4/0x480
<4> [350.101189] ? __pfx_kthread+0x10/0x10
<4> [350.101194] ret_from_fork_asm+0x1a/0x30
<4> [350.101205] </TASK>
<4> [350.101207] irq event stamp: 2060785
<4> [350.101209] hardirqs last enabled at (2060791): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.101214] hardirqs last disabled at (2060796): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.101217] softirqs last enabled at (2059848): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.101221] softirqs last disabled at (2059841): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.101224] ---[ end trace 0000000000000000 ]---
<6> [350.102199] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.102449] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.102457] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.102525] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.102630] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.102739] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.102945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.103035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.103128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.103237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.103323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.103408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.103492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.103578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.103659] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.103754] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.103871] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.104887] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.115502] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.115754] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.116792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.116872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.116948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.117024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.117099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.117186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.117261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.117339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.117416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.117491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.117564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.117639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.117713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.117788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.117863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.117937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.118010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.118083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.118161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.118245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.118327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.118405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.118484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.118560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.118635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.118710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.118785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.118862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.118941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.119017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.119094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.119181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.119269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.119355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.119440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.119521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.119602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.119679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.119758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.119834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.119913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.119990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.120066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.120146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.120229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.120308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.120387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.120464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.120542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.120617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.120692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.120771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.120851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.120930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.121008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.121085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.121167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.121242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.121319] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.121435] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.121439] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278479, lrc_seqno=278479, guc_id=0, flags=0x73 in no process [-1]
<7> [350.121442] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.121500] ------------[ cut here ]------------
<4> [350.121501] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.121502] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [350.121573] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.121637] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.121642] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.121645] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.121646] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.121647] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.121653] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.121722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.121724] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.121726] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.121728] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.121729] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.121730] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.121731] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.121733] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [350.121734] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.121736] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.121737] PKRU: 55555554
<4> [350.121738] Call Trace:
<4> [350.121739] <TASK>
<4> [350.121744] ? lock_sync+0x100/0x100
<4> [350.121748] ? lock_release+0xd0/0x2b0
<4> [350.121754] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.121760] process_one_work+0x239/0x760
<4> [350.121766] worker_thread+0x200/0x3f0
<4> [350.121769] ? __pfx_worker_thread+0x10/0x10
<4> [350.121771] kthread+0x10d/0x150
<4> [350.121774] ? __pfx_kthread+0x10/0x10
<4> [350.121778] ret_from_fork+0x3d4/0x480
<4> [350.121780] ? __pfx_kthread+0x10/0x10
<4> [350.121784] ret_from_fork_asm+0x1a/0x30
<4> [350.121791] </TASK>
<4> [350.121792] irq event stamp: 2063923
<4> [350.121793] hardirqs last enabled at (2063929): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.121796] hardirqs last disabled at (2063934): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.121798] softirqs last enabled at (2063054): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.121801] softirqs last disabled at (2063049): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.121803] ---[ end trace 0000000000000000 ]---
<7> [350.122843] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.122951] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.125962] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278481, lrc_seqno=278481, guc_id=0, flags=0x73 in no process [-1]
<7> [350.125967] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.126031] ------------[ cut here ]------------
<4> [350.126032] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.126034] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [350.126110] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.126185] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.126190] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.126193] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.126194] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.126196] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.126201] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.126274] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.126276] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.126278] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.126280] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.126281] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.126282] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.126283] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.126285] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.126286] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.126288] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.126289] PKRU: 55555554
<4> [350.126290] Call Trace:
<4> [350.126291] <TASK>
<4> [350.126296] ? lock_sync+0x100/0x100
<4> [350.126300] ? lock_release+0xd0/0x2b0
<4> [350.126306] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.126311] process_one_work+0x239/0x760
<4> [350.126318] worker_thread+0x200/0x3f0
<4> [350.126321] ? __pfx_worker_thread+0x10/0x10
<4> [350.126323] kthread+0x10d/0x150
<4> [350.126326] ? __pfx_kthread+0x10/0x10
<4> [350.126330] ret_from_fork+0x3d4/0x480
<4> [350.126332] ? __pfx_kthread+0x10/0x10
<4> [350.126335] ret_from_fork_asm+0x1a/0x30
<4> [350.126343] </TASK>
<4> [350.126344] irq event stamp: 1876837
<4> [350.126345] hardirqs last enabled at (1876843): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.126348] hardirqs last disabled at (1876848): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.126350] softirqs last enabled at (1876318): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.126353] softirqs last disabled at (1876313): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.126355] ---[ end trace 0000000000000000 ]---
<6> [350.126357] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.126427] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.126433] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.126711] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.126916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.127007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.127099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.127199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.127284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.127368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.127451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.127536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.127616] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.127710] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.127826] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.128835] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.139146] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.139397] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.140408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.140487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.140563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.140639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.140716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.140792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.140866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.140940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.141014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.141088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.141169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.141247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.141323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.141397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.141471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.141544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.141618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.141691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.141764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.141846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.141926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.142005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.142084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.142168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.142248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.142328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.142405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.142484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.142564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.142642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.142725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.142807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.142892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.142977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.143062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.143145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.143225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.143305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.143385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.143463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.143543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.143621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.143697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.143773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.143853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.143930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.144007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.144085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.144171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.144248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.144324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.144403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.144480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.144556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.144632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.144714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.144792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.144869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.144950] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.145064] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.145068] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278481, lrc_seqno=278481, guc_id=0, flags=0x73 in no process [-1]
<7> [350.145071] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.145128] ------------[ cut here ]------------
<4> [350.145129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.145131] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [350.145215] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.145280] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.145285] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.145288] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.145289] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.145290] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.145295] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.145364] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.145366] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.145368] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.145369] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.145371] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.145372] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.145373] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.145374] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [350.145376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.145377] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.145379] PKRU: 55555554
<4> [350.145380] Call Trace:
<4> [350.145381] <TASK>
<4> [350.145385] ? lock_sync+0x100/0x100
<4> [350.145389] ? lock_release+0xd0/0x2b0
<4> [350.145395] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.145400] process_one_work+0x239/0x760
<4> [350.145406] worker_thread+0x200/0x3f0
<4> [350.145409] ? __pfx_worker_thread+0x10/0x10
<4> [350.145411] kthread+0x10d/0x150
<4> [350.145415] ? __pfx_kthread+0x10/0x10
<4> [350.145418] ret_from_fork+0x3d4/0x480
<4> [350.145420] ? __pfx_kthread+0x10/0x10
<4> [350.145424] ret_from_fork_asm+0x1a/0x30
<4> [350.145431] </TASK>
<4> [350.145432] irq event stamp: 2067107
<4> [350.145433] hardirqs last enabled at (2067113): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.145436] hardirqs last disabled at (2067118): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.145438] softirqs last enabled at (2065996): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.145441] softirqs last disabled at (2065987): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.145443] ---[ end trace 0000000000000000 ]---
<6> [350.145445] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.145516] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.145522] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.146756] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.146861] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.146966] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.147188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.147280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.147373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.147461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.147547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.147631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.147716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.147807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.147891] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.147986] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.148100] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.149259] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.159149] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.159400] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.160431] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.160509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.160585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.160662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.160740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.160817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.160892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.160967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.161040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.161114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.161250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.161323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.161397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.161469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.161542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.161615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.161687] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.161763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.161841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.161924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.162004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.162082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.162187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.162265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.162342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.162418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.162493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.162572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.162652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.162729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.162808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.162886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.162969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.163054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.163145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.163230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.163311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.163388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.163464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.163539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.163618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.163696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.163774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.163851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.163932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.164010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.164087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.164169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.164247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.164321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.164395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.164473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.164548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.164624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.164701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.164782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.164859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.164934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.165012] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.165126] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.165131] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278481, lrc_seqno=278481, guc_id=0, flags=0x73 in no process [-1]
<7> [350.165133] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.166064] ------------[ cut here ]------------
<4> [350.166065] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.166067] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:44/5138
<4> [350.166146] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.166212] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.166217] CPU: 4 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.166220] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.166221] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.166223] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.166228] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.166298] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.166300] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.166302] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.166304] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.166305] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.166306] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.166307] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.166309] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [350.166310] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.166312] CR2: 00007f7965941000 CR3: 000000019fce7004 CR4: 0000000000f72ef0
<4> [350.166313] PKRU: 55555554
<4> [350.166314] Call Trace:
<4> [350.166315] <TASK>
<4> [350.166320] ? lock_sync+0x100/0x100
<4> [350.166324] ? lock_release+0xd0/0x2b0
<4> [350.166330] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.166335] process_one_work+0x239/0x760
<4> [350.166342] worker_thread+0x200/0x3f0
<4> [350.166344] ? __pfx_worker_thread+0x10/0x10
<4> [350.166347] kthread+0x10d/0x150
<4> [350.166350] ? __pfx_kthread+0x10/0x10
<4> [350.166353] ret_from_fork+0x3d4/0x480
<4> [350.166356] ? __pfx_kthread+0x10/0x10
<4> [350.166359] ret_from_fork_asm+0x1a/0x30
<4> [350.166366] </TASK>
<4> [350.166367] irq event stamp: 2034731
<4> [350.166369] hardirqs last enabled at (2034737): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.166372] hardirqs last disabled at (2034742): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.166374] softirqs last enabled at (2033604): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.166377] softirqs last disabled at (2033571): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.166379] ---[ end trace 0000000000000000 ]---
<7> [350.166392] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.166494] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.172059] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278482, lrc_seqno=278482, guc_id=0, flags=0x73 in no process [-1]
<7> [350.172064] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.172130] ------------[ cut here ]------------
<4> [350.172131] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.172133] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [350.172225] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.172291] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.172295] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.172298] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.172300] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.172301] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.172307] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.172378] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.172380] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.172382] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.172384] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.172385] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.172386] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.172388] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.172389] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.172390] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.172392] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.172393] PKRU: 55555554
<4> [350.172395] Call Trace:
<4> [350.172396] <TASK>
<4> [350.172400] ? lock_sync+0x100/0x100
<4> [350.172404] ? lock_release+0xd0/0x2b0
<4> [350.172410] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.172415] process_one_work+0x239/0x760
<4> [350.172422] worker_thread+0x200/0x3f0
<4> [350.172424] ? __pfx_worker_thread+0x10/0x10
<4> [350.172427] kthread+0x10d/0x150
<4> [350.172430] ? __pfx_kthread+0x10/0x10
<4> [350.172433] ret_from_fork+0x3d4/0x480
<4> [350.172436] ? __pfx_kthread+0x10/0x10
<4> [350.172439] ret_from_fork_asm+0x1a/0x30
<4> [350.172446] </TASK>
<4> [350.172447] irq event stamp: 2035721
<4> [350.172448] hardirqs last enabled at (2035727): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.172451] hardirqs last disabled at (2035732): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.172453] softirqs last enabled at (2033604): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.172456] softirqs last disabled at (2033571): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.172458] ---[ end trace 0000000000000000 ]---
<6> [350.172460] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.172531] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.172538] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.172710] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.172920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.173014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.173108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.173242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.173335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.173423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.173508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.173594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.173675] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.173770] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.173888] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.174901] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.185149] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.185400] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.186389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.186469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.186545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.186622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.186699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.186775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.186851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.186925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.186998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.187072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.187235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.187310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.187385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.187459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.187535] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.187611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.187684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.187759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.187831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.187911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.187992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.188070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.188163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.188253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.188331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.188408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.188485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.188563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.188642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.188723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.188804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.188883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.188970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.189054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.189138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.189238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.189328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.189405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.189483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.189560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.189643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.189721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.189800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.189876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.189956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.190032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.190109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.190199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.190287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.190362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.190439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.190520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.190598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.190675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.190750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.190826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.190899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.190972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.191049] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.191176] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.191183] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278482, lrc_seqno=278482, guc_id=0, flags=0x73 in no process [-1]
<7> [350.191187] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.191253] ------------[ cut here ]------------
<4> [350.191254] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.191255] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:26/3887
<4> [350.191327] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.191391] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.191396] CPU: 0 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.191399] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.191400] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.191402] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.191407] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.191478] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.191479] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.191482] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.191483] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.191484] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.191486] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.191487] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.191488] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.191490] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.191491] CR2: 00007f7966135000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.191493] PKRU: 55555554
<4> [350.191494] Call Trace:
<4> [350.191495] <TASK>
<4> [350.191499] ? lock_sync+0x100/0x100
<4> [350.191503] ? lock_release+0xd0/0x2b0
<4> [350.191509] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.191515] process_one_work+0x239/0x760
<4> [350.191521] worker_thread+0x200/0x3f0
<4> [350.191524] ? __pfx_worker_thread+0x10/0x10
<4> [350.191526] kthread+0x10d/0x150
<4> [350.191529] ? __pfx_kthread+0x10/0x10
<4> [350.191532] ret_from_fork+0x3d4/0x480
<4> [350.191534] ? __pfx_kthread+0x10/0x10
<4> [350.191538] ret_from_fork_asm+0x1a/0x30
<4> [350.191545] </TASK>
<4> [350.191546] irq event stamp: 1431801
<4> [350.191547] hardirqs last enabled at (1431807): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.191550] hardirqs last disabled at (1431812): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.191552] softirqs last enabled at (1430986): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.191555] softirqs last disabled at (1430979): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.191557] ---[ end trace 0000000000000000 ]---
<6> [350.191559] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.191630] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.191636] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.192881] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.192986] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.193090] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.193301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.193390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.193482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.193572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.193658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.193743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.193831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.193920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.194002] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.194097] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.194221] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.195228] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.205950] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.206196] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.207227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.207306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.207381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.207458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.207535] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.207610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.207686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.207762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.207837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.207912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.207988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.208064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.208136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.208227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.208302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.208376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.208449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.208522] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.208595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.208675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.208757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.208836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.208917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.208993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.209069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.209146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.209222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.209300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.209379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.209459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.209539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.209618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.209702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.209785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.209868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.209948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.210026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.210102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.210187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.210263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.210340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.210416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.210493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.210567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.210646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.210721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.210797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.210875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.210954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.211031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.211108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.211199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.211279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.211358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.211435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.211513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.211589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.211664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.211742] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.211856] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.211860] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278482, lrc_seqno=278482, guc_id=0, flags=0x73 in no process [-1]
<7> [350.211863] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.211921] ------------[ cut here ]------------
<4> [350.211922] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.211924] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [350.211995] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.212058] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.212062] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.212065] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.212066] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.212068] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.212073] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.212146] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.212148] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.212150] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.212152] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.212153] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.212155] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.212156] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.212158] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [350.212159] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.212161] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.212163] PKRU: 55555554
<4> [350.212164] Call Trace:
<4> [350.212165] <TASK>
<4> [350.212169] ? lock_sync+0x100/0x100
<4> [350.212173] ? lock_release+0xd0/0x2b0
<4> [350.212179] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.212184] process_one_work+0x239/0x760
<4> [350.212191] worker_thread+0x200/0x3f0
<4> [350.212193] ? __pfx_worker_thread+0x10/0x10
<4> [350.212196] kthread+0x10d/0x150
<4> [350.212199] ? __pfx_kthread+0x10/0x10
<4> [350.212203] ret_from_fork+0x3d4/0x480
<4> [350.212205] ? __pfx_kthread+0x10/0x10
<4> [350.212209] ret_from_fork_asm+0x1a/0x30
<4> [350.212216] </TASK>
<4> [350.212217] irq event stamp: 2070733
<4> [350.212218] hardirqs last enabled at (2070739): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.212221] hardirqs last disabled at (2070744): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.212224] softirqs last enabled at (2069740): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.212226] softirqs last disabled at (2069735): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.212228] ---[ end trace 0000000000000000 ]---
<7> [350.213275] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.213378] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.217212] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278483, lrc_seqno=278483, guc_id=0, flags=0x73 in no process [-1]
<7> [350.217217] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.217282] ------------[ cut here ]------------
<4> [350.217283] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.217285] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [350.217361] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.217425] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.217430] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.217433] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.217434] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.217435] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.217441] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.217511] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.217513] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.217516] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.217517] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.217518] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.217519] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.217521] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.217522] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.217524] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.217525] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.217526] PKRU: 55555554
<4> [350.217528] Call Trace:
<4> [350.217529] <TASK>
<4> [350.217533] ? lock_sync+0x100/0x100
<4> [350.217538] ? lock_release+0xd0/0x2b0
<4> [350.217543] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.217549] process_one_work+0x239/0x760
<4> [350.217555] worker_thread+0x200/0x3f0
<4> [350.217558] ? __pfx_worker_thread+0x10/0x10
<4> [350.217560] kthread+0x10d/0x150
<4> [350.217563] ? __pfx_kthread+0x10/0x10
<4> [350.217566] ret_from_fork+0x3d4/0x480
<4> [350.217569] ? __pfx_kthread+0x10/0x10
<4> [350.217572] ret_from_fork_asm+0x1a/0x30
<4> [350.217579] </TASK>
<4> [350.217580] irq event stamp: 2037333
<4> [350.217581] hardirqs last enabled at (2037339): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.217584] hardirqs last disabled at (2037344): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.217586] softirqs last enabled at (2036502): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.217589] softirqs last disabled at (2036497): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.217591] ---[ end trace 0000000000000000 ]---
<6> [350.217593] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.217663] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.217669] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.217853] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.218057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.218154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.218247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.218335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.218421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.218508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.218594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.218681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.218762] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.218856] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.218971] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.219996] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.230150] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.230403] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.231450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.231529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.231605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.231682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.231759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.231836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.231913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.231987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.232060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.232133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.232225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.232297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.232370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.232443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.232515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.232588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.232661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.232735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.232806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.232886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.232966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.233044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.233125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.233222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.233300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.233380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.233458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.233538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.233617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.233695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.233772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.233850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.233934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.234017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.234098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.234187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.234267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.234343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.234419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.234494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.234572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.234649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.234728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.234809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.234892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.234970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.235049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.235124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.235230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.235307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.235382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.235461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.235538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.235614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.235693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.235776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.235854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.235932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.236011] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.236127] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.236132] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278483, lrc_seqno=278483, guc_id=0, flags=0x73 in no process [-1]
<7> [350.236134] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.236227] ------------[ cut here ]------------
<4> [350.236229] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.236230] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [350.236302] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.236366] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.236371] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.236373] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.236375] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.236376] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.236381] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.236451] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.236453] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.236455] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.236457] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.236458] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.236459] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.236460] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.236462] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.236463] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.236465] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.236466] PKRU: 55555554
<4> [350.236467] Call Trace:
<4> [350.236468] <TASK>
<4> [350.236473] ? lock_sync+0x100/0x100
<4> [350.236477] ? lock_release+0xd0/0x2b0
<4> [350.236483] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.236488] process_one_work+0x239/0x760
<4> [350.236494] worker_thread+0x200/0x3f0
<4> [350.236497] ? __pfx_worker_thread+0x10/0x10
<4> [350.236500] kthread+0x10d/0x150
<4> [350.236502] ? __pfx_kthread+0x10/0x10
<4> [350.236506] ret_from_fork+0x3d4/0x480
<4> [350.236508] ? __pfx_kthread+0x10/0x10
<4> [350.236512] ret_from_fork_asm+0x1a/0x30
<4> [350.236519] </TASK>
<4> [350.236520] irq event stamp: 2040513
<4> [350.236521] hardirqs last enabled at (2040519): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.236524] hardirqs last disabled at (2040524): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.236526] softirqs last enabled at (2039746): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.236529] softirqs last disabled at (2039739): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.236531] ---[ end trace 0000000000000000 ]---
<6> [350.236533] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.236602] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.236609] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.237850] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.237956] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.238061] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.238275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.238369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.238464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.238553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.238638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.238723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.238807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.238895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.238977] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.239072] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.239192] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.240202] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.250147] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.250399] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.251434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.251514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.251591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.251668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.251744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.251819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.251894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.251969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.252043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.252120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.252205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.252281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.252355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.252430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.252502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.252575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.252647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.252719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.252790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.252872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.252956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.253035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.253115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.253200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.253276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.253353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.253429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.253507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.253588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.253670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.253750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.253830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.253915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.253998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.254082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.254164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.254245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.254321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.254398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.254473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.254552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.254632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.254710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.254786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.254864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.254939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.255016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.255091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.255181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.255257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.255337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.255419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.255497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.255577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.255653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.255732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.255807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.255883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.255961] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.256076] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.256081] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278483, lrc_seqno=278483, guc_id=0, flags=0x73 in no process [-1]
<7> [350.256083] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.256145] ------------[ cut here ]------------
<4> [350.256146] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.256148] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:38/5132
<4> [350.256218] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.256283] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.256287] CPU: 10 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.256290] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.256292] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.256293] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.256299] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.256368] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.256370] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.256373] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.256374] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.256375] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.256377] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.256378] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.256379] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [350.256381] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.256382] CR2: 00007400ed696450 CR3: 000000011fc71001 CR4: 0000000000f72ef0
<4> [350.256384] PKRU: 55555554
<4> [350.256385] Call Trace:
<4> [350.256386] <TASK>
<4> [350.256390] ? lock_sync+0x100/0x100
<4> [350.256395] ? lock_release+0xd0/0x2b0
<4> [350.256400] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.256405] process_one_work+0x239/0x760
<4> [350.256412] worker_thread+0x200/0x3f0
<4> [350.256414] ? __pfx_worker_thread+0x10/0x10
<4> [350.256417] kthread+0x10d/0x150
<4> [350.256419] ? __pfx_kthread+0x10/0x10
<4> [350.256423] ret_from_fork+0x3d4/0x480
<4> [350.256425] ? __pfx_kthread+0x10/0x10
<4> [350.256428] ret_from_fork_asm+0x1a/0x30
<4> [350.256436] </TASK>
<4> [350.256437] irq event stamp: 1880385
<4> [350.256438] hardirqs last enabled at (1880391): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.256441] hardirqs last disabled at (1880396): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.256443] softirqs last enabled at (1879618): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.256445] softirqs last disabled at (1879611): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.256448] ---[ end trace 0000000000000000 ]---
<7> [350.257500] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.257609] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.258027] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278484, lrc_seqno=278484, guc_id=0, flags=0x73 in no process [-1]
<7> [350.258031] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.258119] ------------[ cut here ]------------
<4> [350.258120] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [350.258138] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:667]
<4> [350.258122] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [350.258224] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.258297] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.258301] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.258305] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.258306] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.258307] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.258312] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.258385] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.258386] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.258389] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.258390] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.258391] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.258393] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.258394] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.258395] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.258397] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.258398] CR2: 00007f7966135000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.258399] PKRU: 55555554
<4> [350.258401] Call Trace:
<4> [350.258402] <TASK>
<4> [350.258406] ? lock_sync+0x100/0x100
<4> [350.258410] ? lock_release+0xd0/0x2b0
<4> [350.258416] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.258421] process_one_work+0x239/0x760
<4> [350.258428] worker_thread+0x200/0x3f0
<4> [350.258430] ? __pfx_worker_thread+0x10/0x10
<4> [350.258433] kthread+0x10d/0x150
<4> [350.258436] ? __pfx_kthread+0x10/0x10
<4> [350.258439] ret_from_fork+0x3d4/0x480
<4> [350.258442] ? __pfx_kthread+0x10/0x10
<4> [350.258445] ret_from_fork_asm+0x1a/0x30
<4> [350.258452] </TASK>
<4> [350.258453] irq event stamp: 1881325
<4> [350.258454] hardirqs last enabled at (1881331): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.258457] hardirqs last disabled at (1881336): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.258459] softirqs last enabled at (1880576): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.258462] softirqs last disabled at (1880569): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.258464] ---[ end trace 0000000000000000 ]---
<6> [350.258474] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.258547] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.258558] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.258708] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.258925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.259014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.259107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.259212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.259299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.259385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.259474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.259564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.259647] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.259741] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.259859] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.260955] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.271497] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.271748] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.272791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.272869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.272945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.273022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.273099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.273184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.273258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.273335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.273411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.273486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.273562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.273636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.273709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.273782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.273854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.273925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.273997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.274070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.274151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.274235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.274315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.274394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.274477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.274556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.274633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.274709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.274784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.274861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.274939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.275015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.275092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.275179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.275268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.275354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.275440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.275522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.275603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.275681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.275761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.275840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.275921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.276000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.276077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.276159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.276241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.276321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.276399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.276475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.276553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.276629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.276704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.276782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.276858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.276934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.277012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.277092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.277179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.277257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.277336] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.277453] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.277458] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278484, lrc_seqno=278484, guc_id=0, flags=0x73 in no process [-1]
<7> [350.277460] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.277518] ------------[ cut here ]------------
<4> [350.277519] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.277520] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [350.277590] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.277654] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.277658] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.277661] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.277662] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.277664] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.277669] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.277738] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.277740] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.277742] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.277744] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.277745] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.277746] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.277747] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.277748] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [350.277750] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.277751] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [350.277753] PKRU: 55555554
<4> [350.277754] Call Trace:
<4> [350.277755] <TASK>
<4> [350.277759] ? lock_sync+0x100/0x100
<4> [350.277764] ? lock_release+0xd0/0x2b0
<4> [350.277769] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.277775] process_one_work+0x239/0x760
<4> [350.277781] worker_thread+0x200/0x3f0
<4> [350.277784] ? __pfx_worker_thread+0x10/0x10
<4> [350.277786] kthread+0x10d/0x150
<4> [350.277789] ? __pfx_kthread+0x10/0x10
<4> [350.277793] ret_from_fork+0x3d4/0x480
<4> [350.277795] ? __pfx_kthread+0x10/0x10
<4> [350.277798] ret_from_fork_asm+0x1a/0x30
<4> [350.277805] </TASK>
<4> [350.277806] irq event stamp: 1435491
<4> [350.277807] hardirqs last enabled at (1435497): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.277810] hardirqs last disabled at (1435502): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.277813] softirqs last enabled at (1434534): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.277815] softirqs last disabled at (1434527): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.277817] ---[ end trace 0000000000000000 ]---
<6> [350.277819] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.277889] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.277895] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.279126] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.279243] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.279364] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.279575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.279666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.279760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.279850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.279936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.280021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.280108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.280209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.280295] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.280391] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.280506] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.281515] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.292153] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.292408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.293432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.293511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.293588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.293666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.293741] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.293815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.293890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.293964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.294040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.294116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.294202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.294276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.294349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.294423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.294497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.294570] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.294642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.294714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.294786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.294866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.294945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.295023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.295104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.295207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.295286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.295363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.295439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.295518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.295597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.295675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.295752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.295829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.295912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.295996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.296085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.296176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.296257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.296334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.296411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.296487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.296566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.296644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.296721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.296797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.296878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.296958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.297035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.297112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.297201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.297303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.297441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.297563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.297679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.297797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.297914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.298033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.298153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.298269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.298382] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.298497] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.298502] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278484, lrc_seqno=278484, guc_id=0, flags=0x73 in no process [-1]
<7> [350.298504] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.298565] ------------[ cut here ]------------
<4> [350.298566] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.298567] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [350.298641] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.298705] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.298710] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.298713] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.298714] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.298715] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.298721] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.298792] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.298793] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.298796] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.298797] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.298799] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.298800] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.298801] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.298802] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.298804] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.298805] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.298807] PKRU: 55555554
<4> [350.298808] Call Trace:
<4> [350.298809] <TASK>
<4> [350.298813] ? lock_sync+0x100/0x100
<4> [350.298818] ? lock_release+0xd0/0x2b0
<4> [350.298824] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.298829] process_one_work+0x239/0x760
<4> [350.298836] worker_thread+0x200/0x3f0
<4> [350.298838] ? __pfx_worker_thread+0x10/0x10
<4> [350.298841] kthread+0x10d/0x150
<4> [350.298843] ? __pfx_kthread+0x10/0x10
<4> [350.298847] ret_from_fork+0x3d4/0x480
<4> [350.298849] ? __pfx_kthread+0x10/0x10
<4> [350.298852] ret_from_fork_asm+0x1a/0x30
<4> [350.298860] </TASK>
<4> [350.298861] irq event stamp: 2044107
<4> [350.298862] hardirqs last enabled at (2044113): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.298865] hardirqs last disabled at (2044118): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.298867] softirqs last enabled at (2043184): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.298870] softirqs last disabled at (2043179): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.298872] ---[ end trace 0000000000000000 ]---
<7> [350.300197] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.300302] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.300789] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278493, lrc_seqno=278493, guc_id=0, flags=0x73 in no process [-1]
<7> [350.300792] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.300854] ------------[ cut here ]------------
<4> [350.300855] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.300856] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [350.300929] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.300995] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.300999] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.301002] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.301003] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.301005] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.301010] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.301080] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.301082] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.301084] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.301085] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.301087] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.301088] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.301089] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.301090] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [350.301092] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.301093] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [350.301095] PKRU: 55555554
<4> [350.301096] Call Trace:
<4> [350.301097] <TASK>
<4> [350.301101] ? lock_sync+0x100/0x100
<4> [350.301105] ? lock_release+0xd0/0x2b0
<4> [350.301111] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.301116] process_one_work+0x239/0x760
<4> [350.301123] worker_thread+0x200/0x3f0
<4> [350.301125] ? __pfx_worker_thread+0x10/0x10
<4> [350.301128] kthread+0x10d/0x150
<4> [350.301131] ? __pfx_kthread+0x10/0x10
<4> [350.301134] ret_from_fork+0x3d4/0x480
<4> [350.301136] ? __pfx_kthread+0x10/0x10
<4> [350.301144] ret_from_fork_asm+0x1a/0x30
<4> [350.301151] </TASK>
<4> [350.301153] irq event stamp: 1437153
<4> [350.301154] hardirqs last enabled at (1437159): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.301157] hardirqs last disabled at (1437164): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.301160] softirqs last enabled at (1435846): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [350.301162] softirqs last disabled at (1435844): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [350.301165] ---[ end trace 0000000000000000 ]---
<6> [350.301167] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.301238] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.301244] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.301643] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.301846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.301934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.302027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.302114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.302210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.302309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.302393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.302479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.302561] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.302658] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.302791] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.303867] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.314150] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.314404] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.315426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.315508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.315585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.315664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.315741] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.315816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.315892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.315968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.316043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.316121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.316213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.316289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.316363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.316437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.316510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.316581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.316654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.316726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.316798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.316879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.316960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.317040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.317124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.317216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.317295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.317372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.317448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.317527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.317606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.317684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.317763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.317841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.317925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.318012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.318099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.318190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.318272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.318351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.318429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.318508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.318590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.318668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.318745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.318820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.318899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.318976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.319052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.319126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.319218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.319297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.319401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.319534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.319649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.319766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.319878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.319999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.320120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.320265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.320385] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.320516] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.320521] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278493, lrc_seqno=278493, guc_id=0, flags=0x73 in no process [-1]
<7> [350.320523] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.320582] ------------[ cut here ]------------
<4> [350.320583] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.320585] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [350.320657] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.320719] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.320724] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.320727] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.320728] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.320730] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.320735] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.320805] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.320807] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.320809] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.320810] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.320812] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.320813] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.320814] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.320816] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.320817] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.320818] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.320820] PKRU: 55555554
<4> [350.320821] Call Trace:
<4> [350.320822] <TASK>
<4> [350.320826] ? lock_sync+0x100/0x100
<4> [350.320831] ? lock_release+0xd0/0x2b0
<4> [350.320836] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.320842] process_one_work+0x239/0x760
<4> [350.320848] worker_thread+0x200/0x3f0
<4> [350.320851] ? __pfx_worker_thread+0x10/0x10
<4> [350.320853] kthread+0x10d/0x150
<4> [350.320856] ? __pfx_kthread+0x10/0x10
<4> [350.320860] ret_from_fork+0x3d4/0x480
<4> [350.320862] ? __pfx_kthread+0x10/0x10
<4> [350.320865] ret_from_fork_asm+0x1a/0x30
<4> [350.320872] </TASK>
<4> [350.320873] irq event stamp: 2047411
<4> [350.320875] hardirqs last enabled at (2047417): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.320877] hardirqs last disabled at (2047422): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.320880] softirqs last enabled at (2046488): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.320882] softirqs last disabled at (2046483): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.320885] ---[ end trace 0000000000000000 ]---
<6> [350.320886] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.320958] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.320964] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.321373] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.321577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.321667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.321757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.321843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.321929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.322013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.322098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.322197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.322284] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.322380] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.322496] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.323503] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.334151] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.334405] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.335415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.335496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.335575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.335654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.335731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.335807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.335882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.335957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.336031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.336107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.336194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.336268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.336341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.336415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.336489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.336566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.336643] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.336720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.336795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.336876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.336956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.337033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.337115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.337214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.337293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.337369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.337445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.337523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.337604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.337685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.337764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.337842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.337925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.338008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.338091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.338179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.338259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.338335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.338411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.338486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.338564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.338641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.338720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.338796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.338876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.338952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.339029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.339104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.339192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.339272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.339350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.339431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.339508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.339586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.339661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.339738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.339814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.339890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.339968] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.340084] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.340088] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278493, lrc_seqno=278493, guc_id=0, flags=0x73 in no process [-1]
<7> [350.340090] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.341006] ------------[ cut here ]------------
<4> [350.341007] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.341008] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [350.341081] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.341151] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.341156] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.341159] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.341160] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.341162] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.341168] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.341237] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.341239] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.341241] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.341243] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.341244] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.341245] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.341246] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.341248] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.341249] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.341251] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.341252] PKRU: 55555554
<4> [350.341253] Call Trace:
<4> [350.341254] <TASK>
<4> [350.341258] ? lock_sync+0x100/0x100
<4> [350.341263] ? lock_release+0xd0/0x2b0
<4> [350.341268] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.341274] process_one_work+0x239/0x760
<4> [350.341280] worker_thread+0x200/0x3f0
<4> [350.341282] ? __pfx_worker_thread+0x10/0x10
<4> [350.341285] kthread+0x10d/0x150
<4> [350.341288] ? __pfx_kthread+0x10/0x10
<4> [350.341291] ret_from_fork+0x3d4/0x480
<4> [350.341293] ? __pfx_kthread+0x10/0x10
<4> [350.341297] ret_from_fork_asm+0x1a/0x30
<4> [350.341304] </TASK>
<4> [350.341305] irq event stamp: 2050547
<4> [350.341306] hardirqs last enabled at (2050553): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.341309] hardirqs last disabled at (2050558): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.341311] softirqs last enabled at (2049430): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.341314] softirqs last disabled at (2049421): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.341316] ---[ end trace 0000000000000000 ]---
<7> [350.341329] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.341433] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.443703] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278501, lrc_seqno=278501, guc_id=0, flags=0x73 in no process [-1]
<7> [350.443707] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.443789] ------------[ cut here ]------------
<4> [350.443791] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.443793] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:26/3887
<4> [350.443860] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.443944] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.443949] CPU: 0 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.443951] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.443952] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.443954] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.443960] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.444025] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.444026] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.444028] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.444030] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.444031] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.444032] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.444033] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.444034] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.444036] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.444037] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.444038] PKRU: 55555554
<4> [350.444039] Call Trace:
<4> [350.444040] <TASK>
<4> [350.444044] ? lock_sync+0x100/0x100
<4> [350.444048] ? lock_release+0xd0/0x2b0
<4> [350.444053] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.444058] process_one_work+0x239/0x760
<4> [350.444064] worker_thread+0x200/0x3f0
<4> [350.444066] ? __pfx_worker_thread+0x10/0x10
<4> [350.444068] kthread+0x10d/0x150
<4> [350.444071] ? __pfx_kthread+0x10/0x10
<4> [350.444074] ret_from_fork+0x3d4/0x480
<4> [350.444076] ? __pfx_kthread+0x10/0x10
<4> [350.444079] ret_from_fork_asm+0x1a/0x30
<4> [350.444085] </TASK>
<4> [350.444086] irq event stamp: 1440279
<4> [350.444087] hardirqs last enabled at (1440285): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.444090] hardirqs last disabled at (1440290): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.444092] softirqs last enabled at (1438264): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [350.444094] softirqs last disabled at (1438262): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [350.444096] ---[ end trace 0000000000000000 ]---
<6> [350.444098] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.444180] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.444189] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.444609] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.444833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.444946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.445051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.445156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.445251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.445352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.445429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.445506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.445581] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.445667] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.445774] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.446989] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.457706] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.457948] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.459029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.459099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.459183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.459266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.459338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.459408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.459479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.459549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.459617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.459686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.459754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.459821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.459889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.459956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.460022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.460088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.460165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.460263] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.460331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.460404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.460478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.460548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.460620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.460690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.460759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.460826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.460897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.460970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.461041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.461111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.461204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.461291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.461367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.461441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.461518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.461591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.461667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.461740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.461814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.461886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.461962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.462038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.462112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.462205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.462299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.462374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.462450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.462524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.462601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.462678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.462754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.462834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.462914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.462999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.463083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.463181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.463277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.463360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.463447] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.463569] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.463573] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278501, lrc_seqno=278501, guc_id=0, flags=0x73 in no process [-1]
<7> [350.463576] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.463641] ------------[ cut here ]------------
<4> [350.463642] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.463644] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:26/3887
<4> [350.463721] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.463791] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.463796] CPU: 0 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.463799] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.463801] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.463802] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.463808] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.463884] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.463886] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.463889] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.463890] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.463892] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.463893] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.463895] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.463896] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.463898] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.463900] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.463901] PKRU: 55555554
<4> [350.463903] Call Trace:
<4> [350.463904] <TASK>
<4> [350.463908] ? lock_sync+0x100/0x100
<4> [350.463913] ? lock_release+0xd0/0x2b0
<4> [350.463919] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.463926] process_one_work+0x239/0x760
<4> [350.463933] worker_thread+0x200/0x3f0
<4> [350.463936] ? __pfx_worker_thread+0x10/0x10
<4> [350.463938] kthread+0x10d/0x150
<4> [350.463942] ? __pfx_kthread+0x10/0x10
<4> [350.463946] ret_from_fork+0x3d4/0x480
<4> [350.463948] ? __pfx_kthread+0x10/0x10
<4> [350.463952] ret_from_fork_asm+0x1a/0x30
<4> [350.463960] </TASK>
<4> [350.463962] irq event stamp: 1443401
<4> [350.463963] hardirqs last enabled at (1443407): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.463966] hardirqs last disabled at (1443412): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.463969] softirqs last enabled at (1442444): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.463972] softirqs last disabled at (1442437): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.463974] ---[ end trace 0000000000000000 ]---
<6> [350.463976] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.464057] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.464064] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.465315] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.465446] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.465573] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.465806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.465914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.466025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.466134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.466264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.466361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.466456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.466554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.466645] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.466750] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.466875] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.467917] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.478151] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.478415] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.479402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.479483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.479560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.479637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.479713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.479788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.479864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.479939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.480013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.480087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.480171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.480247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.480321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.480396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.480469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.480543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.480617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.480689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.480762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.480842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.480922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.481001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.481083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.481169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.481247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.481324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.481399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.481477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.481554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.481631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.481709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.481789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.481873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.481957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.482042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.482127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.482220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.482299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.482376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.482451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.482530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.482609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.482699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.482779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.482860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.482938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.483017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.483095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.483200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.483338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.483430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.483508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.483583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.483659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.483736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.483814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.483890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.483966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.484048] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.484173] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.484177] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278501, lrc_seqno=278501, guc_id=0, flags=0x73 in no process [-1]
<7> [350.484180] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.484238] ------------[ cut here ]------------
<4> [350.484240] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.484241] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [350.484312] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.484376] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.484381] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.484384] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.484385] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.484386] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.484392] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.484461] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.484463] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.484465] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.484467] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.484468] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.484469] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.484470] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.484472] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [350.484474] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.484475] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.484476] PKRU: 55555554
<4> [350.484478] Call Trace:
<4> [350.484479] <TASK>
<4> [350.484483] ? lock_sync+0x100/0x100
<4> [350.484487] ? lock_release+0xd0/0x2b0
<4> [350.484493] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.484498] process_one_work+0x239/0x760
<4> [350.484505] worker_thread+0x200/0x3f0
<4> [350.484507] ? __pfx_worker_thread+0x10/0x10
<4> [350.484510] kthread+0x10d/0x150
<4> [350.484513] ? __pfx_kthread+0x10/0x10
<4> [350.484516] ret_from_fork+0x3d4/0x480
<4> [350.484518] ? __pfx_kthread+0x10/0x10
<4> [350.484522] ret_from_fork_asm+0x1a/0x30
<4> [350.484529] </TASK>
<4> [350.484530] irq event stamp: 2074513
<4> [350.484531] hardirqs last enabled at (2074519): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.484534] hardirqs last disabled at (2074524): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.484536] softirqs last enabled at (2073504): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.484539] softirqs last disabled at (2073497): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.484541] ---[ end trace 0000000000000000 ]---
<7> [350.485602] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.485708] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.488991] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278503, lrc_seqno=278503, guc_id=0, flags=0x73 in no process [-1]
<7> [350.488993] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.489054] ------------[ cut here ]------------
<4> [350.489055] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.489056] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [350.489129] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.489218] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.489222] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.489225] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.489226] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.489228] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.489232] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.489303] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.489304] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.489307] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.489308] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.489309] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.489311] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.489312] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.489313] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [350.489315] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.489316] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.489317] PKRU: 55555554
<4> [350.489319] Call Trace:
<4> [350.489320] <TASK>
<4> [350.489323] ? lock_sync+0x100/0x100
<4> [350.489328] ? lock_release+0xd0/0x2b0
<4> [350.489333] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.489339] process_one_work+0x239/0x760
<4> [350.489345] worker_thread+0x200/0x3f0
<4> [350.489347] ? __pfx_worker_thread+0x10/0x10
<4> [350.489350] kthread+0x10d/0x150
<4> [350.489353] ? __pfx_kthread+0x10/0x10
<4> [350.489356] ret_from_fork+0x3d4/0x480
<4> [350.489358] ? __pfx_kthread+0x10/0x10
<4> [350.489363] ret_from_fork_asm+0x1a/0x30
<4> [350.489376] </TASK>
<4> [350.489378] irq event stamp: 2075445
<4> [350.489380] hardirqs last enabled at (2075451): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.489384] hardirqs last disabled at (2075456): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.489387] softirqs last enabled at (2074786): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.489390] softirqs last disabled at (2074779): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.489392] ---[ end trace 0000000000000000 ]---
<6> [350.489394] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.489467] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.489474] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.489920] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.490124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.490315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.490416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.490503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.490590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.490677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.490766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.490856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.490939] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.491034] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.491154] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.492212] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.502935] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.503230] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.504271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.504350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.504425] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.504501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.504575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.504649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.504725] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.504800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.504878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.504954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.505028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.505101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.505184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.505257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.505330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.505404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.505481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.505557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.505632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.505715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.505796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.505875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.505954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.506031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.506116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.506226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.506306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.506388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.506468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.506547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.506626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.506733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.506849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.506933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.507016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.507096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.507209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.507288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.507368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.507445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.507525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.507603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.507679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.507754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.507834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.507909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.507989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.508068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.508152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.508228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.508304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.508382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.508466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.508543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.508619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.508696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.508774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.508853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.508934] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.509052] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.509056] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278503, lrc_seqno=278503, guc_id=0, flags=0x73 in no process [-1]
<7> [350.509058] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.509116] ------------[ cut here ]------------
<4> [350.509118] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.509119] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [350.509200] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.509265] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.509270] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.509272] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.509273] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.509275] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.509280] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.509350] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.509352] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [350.509354] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.509355] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.509357] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.509358] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.509359] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.509360] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [350.509362] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.509363] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.509365] PKRU: 55555554
<4> [350.509366] Call Trace:
<4> [350.509367] <TASK>
<4> [350.509371] ? lock_sync+0x100/0x100
<4> [350.509375] ? lock_release+0xd0/0x2b0
<4> [350.509381] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.509387] process_one_work+0x239/0x760
<4> [350.509393] worker_thread+0x200/0x3f0
<4> [350.509396] ? __pfx_worker_thread+0x10/0x10
<4> [350.509398] kthread+0x10d/0x150
<4> [350.509401] ? __pfx_kthread+0x10/0x10
<4> [350.509404] ret_from_fork+0x3d4/0x480
<4> [350.509407] ? __pfx_kthread+0x10/0x10
<4> [350.509410] ret_from_fork_asm+0x1a/0x30
<4> [350.509417] </TASK>
<4> [350.509418] irq event stamp: 2078563
<4> [350.509420] hardirqs last enabled at (2078569): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.509422] hardirqs last disabled at (2078574): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.509425] softirqs last enabled at (2077464): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.509427] softirqs last disabled at (2077455): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.509429] ---[ end trace 0000000000000000 ]---
<6> [350.509431] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.509500] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.509506] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.510740] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.510844] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.510949] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.511174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.511274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.511374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.511462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.511548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.511632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.511718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.511805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.511889] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.511986] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.512103] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.513125] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.523863] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.524114] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.525155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.525232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.525310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.525388] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.525464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.525539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.525614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.525688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.525761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.525835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.525909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.525983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.526056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.526129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.526214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.526287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.526362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.526438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.526514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.526596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.526676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.526754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.526833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.526909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.526985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.527061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.527144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.527226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.527306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.527384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.527462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.527540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.527623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.527705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.527788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.527867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.527945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.528022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.528101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.528187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.528268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.528346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.528422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.528498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.528577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.528653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.528728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.528804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.528882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.528961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.529038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.529118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.529203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.529280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.529355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.529432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.529509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.529584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.529664] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.529780] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.529785] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278503, lrc_seqno=278503, guc_id=0, flags=0x73 in no process [-1]
<7> [350.529787] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.529846] ------------[ cut here ]------------
<4> [350.529847] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.529848] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:44/5138
<4> [350.529921] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.529986] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.529990] CPU: 4 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.529993] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.529995] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.529996] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.530002] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.530071] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.530073] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [350.530075] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.530077] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.530078] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.530079] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.530080] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.530081] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [350.530083] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.530084] CR2: 00007f796717f000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.530086] PKRU: 55555554
<4> [350.530087] Call Trace:
<4> [350.530088] <TASK>
<4> [350.530092] ? lock_sync+0x100/0x100
<4> [350.530097] ? lock_release+0xd0/0x2b0
<4> [350.530102] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.530108] process_one_work+0x239/0x760
<4> [350.530114] worker_thread+0x200/0x3f0
<4> [350.530117] ? __pfx_worker_thread+0x10/0x10
<4> [350.530119] kthread+0x10d/0x150
<4> [350.530122] ? __pfx_kthread+0x10/0x10
<4> [350.530126] ret_from_fork+0x3d4/0x480
<4> [350.530128] ? __pfx_kthread+0x10/0x10
<4> [350.530131] ret_from_fork_asm+0x1a/0x30
<4> [350.530139] </TASK>
<4> [350.530140] irq event stamp: 2054039
<4> [350.530145] hardirqs last enabled at (2054047): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.530148] hardirqs last disabled at (2054052): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.530150] softirqs last enabled at (2053144): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.530153] softirqs last disabled at (2053137): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.530155] ---[ end trace 0000000000000000 ]---
<7> [350.531226] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.531381] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.535568] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278504, lrc_seqno=278504, guc_id=0, flags=0x73 in no process [-1]
<7> [350.535573] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.535643] ------------[ cut here ]------------
<4> [350.535644] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.535646] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [350.535721] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.535785] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.535790] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.535793] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.535794] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.535796] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.535801] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.535873] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.535875] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.535877] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.535879] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.535880] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.535881] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.535883] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.535884] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.535885] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.535887] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.535888] PKRU: 55555554
<4> [350.535889] Call Trace:
<4> [350.535890] <TASK>
<4> [350.535895] ? lock_sync+0x100/0x100
<4> [350.535899] ? lock_release+0xd0/0x2b0
<4> [350.535905] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.535910] process_one_work+0x239/0x760
<4> [350.535916] worker_thread+0x200/0x3f0
<4> [350.535919] ? __pfx_worker_thread+0x10/0x10
<4> [350.535921] kthread+0x10d/0x150
<4> [350.535924] ? __pfx_kthread+0x10/0x10
<4> [350.535928] ret_from_fork+0x3d4/0x480
<4> [350.535930] ? __pfx_kthread+0x10/0x10
<4> [350.535933] ret_from_fork_asm+0x1a/0x30
<4> [350.535940] </TASK>
<4> [350.535941] irq event stamp: 1883347
<4> [350.535943] hardirqs last enabled at (1883353): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.535945] hardirqs last disabled at (1883358): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.535948] softirqs last enabled at (1882012): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [350.535950] softirqs last disabled at (1882010): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [350.535952] ---[ end trace 0000000000000000 ]---
<6> [350.535954] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.536068] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.536079] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.536261] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.536507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.536637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.536774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.536903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.537029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.537154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.537245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.537335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.537418] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.537516] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.537635] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.538663] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.549388] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.549639] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.550674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.550754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.550833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.550910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.550986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.551060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.551134] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.551225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.551299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.551373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.551445] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.551519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.551592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.551666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.551739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.551817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.551894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.551968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.552042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.552124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.552218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.552298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.552377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.552454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.552571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.552703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.552820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.552935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.553055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.553187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.553308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.553422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.553515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.553603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.553690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.553770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.553849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.553925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.554001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.554075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.554160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.554237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.554314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.554388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.554467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.554544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.554627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.554707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.554787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.554864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.554939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.555018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.555096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.555188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.555268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.555347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.555423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.555499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.555577] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.555695] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.555700] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278504, lrc_seqno=278504, guc_id=0, flags=0x73 in no process [-1]
<7> [350.555702] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.555760] ------------[ cut here ]------------
<4> [350.555762] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.555763] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:38/5132
<4> [350.555836] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.555899] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.555903] CPU: 6 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.555906] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.555908] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.555909] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.555914] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.555984] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.555986] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.555988] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.555989] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.555991] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.555992] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.555993] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.555995] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [350.555996] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.555998] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [350.555999] PKRU: 55555554
<4> [350.556000] Call Trace:
<4> [350.556001] <TASK>
<4> [350.556005] ? lock_sync+0x100/0x100
<4> [350.556010] ? lock_release+0xd0/0x2b0
<4> [350.556015] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.556021] process_one_work+0x239/0x760
<4> [350.556027] worker_thread+0x200/0x3f0
<4> [350.556029] ? __pfx_worker_thread+0x10/0x10
<4> [350.556032] kthread+0x10d/0x150
<4> [350.556035] ? __pfx_kthread+0x10/0x10
<4> [350.556038] ret_from_fork+0x3d4/0x480
<4> [350.556040] ? __pfx_kthread+0x10/0x10
<4> [350.556044] ret_from_fork_asm+0x1a/0x30
<4> [350.556051] </TASK>
<4> [350.556052] irq event stamp: 1886465
<4> [350.556053] hardirqs last enabled at (1886471): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.556056] hardirqs last disabled at (1886476): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.556058] softirqs last enabled at (1885394): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.556061] softirqs last disabled at (1885387): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.556063] ---[ end trace 0000000000000000 ]---
<6> [350.556065] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.556135] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.556146] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.557386] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.557491] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.557597] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.557805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.557897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.557991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.558079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.558181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.558271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.558356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.558443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.558525] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.558619] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.558737] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.559750] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.570153] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.570405] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.571386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.571466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.571542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.571621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.571698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.571773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.571848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.571921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.571994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.572066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.572138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.572230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.572316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.572393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.572468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.572542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.572615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.572687] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.572760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.572841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.572923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.573001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.573080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.573166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.573253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.573329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.573404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.573482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.573560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.573638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.573717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.573795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.573878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.573961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.574045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.574125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.574221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.574310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.574389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.574466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.574546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.574623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.574699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.574774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.574855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.574930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.575006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.575080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.575164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.575250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.575331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.575411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.575488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.575565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.575640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.575718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.575793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.575869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.575947] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.576062] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.576066] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278504, lrc_seqno=278504, guc_id=0, flags=0x73 in no process [-1]
<7> [350.576069] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.576128] ------------[ cut here ]------------
<4> [350.576129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.576131] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:26/3887
<4> [350.576220] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.576309] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.576314] CPU: 0 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.576317] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.576318] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.576320] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.576325] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.576395] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.576397] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.576399] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.576401] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.576402] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.576404] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.576405] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.576406] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.576408] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.576409] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.576411] PKRU: 55555554
<4> [350.576412] Call Trace:
<4> [350.576413] <TASK>
<4> [350.576417] ? lock_sync+0x100/0x100
<4> [350.576421] ? lock_release+0xd0/0x2b0
<4> [350.576427] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.576432] process_one_work+0x239/0x760
<4> [350.576439] worker_thread+0x200/0x3f0
<4> [350.576441] ? __pfx_worker_thread+0x10/0x10
<4> [350.576444] kthread+0x10d/0x150
<4> [350.576447] ? __pfx_kthread+0x10/0x10
<4> [350.576450] ret_from_fork+0x3d4/0x480
<4> [350.576452] ? __pfx_kthread+0x10/0x10
<4> [350.576456] ret_from_fork_asm+0x1a/0x30
<4> [350.576463] </TASK>
<4> [350.576464] irq event stamp: 1447819
<4> [350.576465] hardirqs last enabled at (1447825): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.576468] hardirqs last disabled at (1447830): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.576470] softirqs last enabled at (1447070): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.576473] softirqs last disabled at (1447065): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.576475] ---[ end trace 0000000000000000 ]---
<7> [350.577752] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.577873] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [350.581241] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278505, lrc_seqno=278505, guc_id=0, flags=0x73 in no process [-1]
<7> [350.581244] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.581307] ------------[ cut here ]------------
<4> [350.581308] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.581309] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:26/3887
<4> [350.581382] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.581445] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.581450] CPU: 0 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.581452] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.581454] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.581455] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.581460] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.581531] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.581532] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [350.581535] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.581536] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.581537] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.581538] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.581540] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.581541] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.581543] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.581544] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.581545] PKRU: 55555554
<4> [350.581547] Call Trace:
<4> [350.581548] <TASK>
<4> [350.581551] ? lock_sync+0x100/0x100
<4> [350.581556] ? lock_release+0xd0/0x2b0
<4> [350.581561] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.581567] process_one_work+0x239/0x760
<4> [350.581573] worker_thread+0x200/0x3f0
<4> [350.581576] ? __pfx_worker_thread+0x10/0x10
<4> [350.581578] kthread+0x10d/0x150
<4> [350.581581] ? __pfx_kthread+0x10/0x10
<4> [350.581585] ret_from_fork+0x3d4/0x480
<4> [350.581587] ? __pfx_kthread+0x10/0x10
<4> [350.581590] ret_from_fork_asm+0x1a/0x30
<4> [350.581598] </TASK>
<4> [350.581599] irq event stamp: 1448675
<4> [350.581600] hardirqs last enabled at (1448681): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.581602] hardirqs last disabled at (1448686): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.581605] softirqs last enabled at (1447070): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.581607] softirqs last disabled at (1447065): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.581609] ---[ end trace 0000000000000000 ]---
<6> [350.581611] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.581680] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.581687] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.581860] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.582068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.582162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.582258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.582347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.582433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.582517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.582601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.582687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.582769] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.582863] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.582979] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.583984] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.594149] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.594401] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.595408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.595488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.595564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.595641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.595718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.595794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.595874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.595950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.596025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.596099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.596184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.596257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.596331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.596405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.596478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.596551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.596624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.596698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.596771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.596851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.596932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.597013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.597094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.597178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.597254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.597330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.597407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.597485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.597565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.597643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.597722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.597799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.597883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.597966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.598050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.598131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.598220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.598296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.598375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.598454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.598535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.598613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.598689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.598765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.598845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.598921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.598998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.599074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.599153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.599229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.599303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.599382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.599458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.599532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.599607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.599686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.599764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.599842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.599924] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.600044] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.600048] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278505, lrc_seqno=278505, guc_id=0, flags=0x73 in no process [-1]
<7> [350.600051] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.600110] ------------[ cut here ]------------
<4> [350.600111] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.600113] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:38/5132
<4> [350.600200] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.600264] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.600269] CPU: 8 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.600272] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.600273] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.600275] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.600280] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.600351] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.600352] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.600355] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.600356] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.600357] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.600358] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.600360] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.600361] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [350.600363] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.600364] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [350.600365] PKRU: 55555554
<4> [350.600367] Call Trace:
<4> [350.600368] <TASK>
<4> [350.600372] ? lock_sync+0x100/0x100
<4> [350.600376] ? lock_release+0xd0/0x2b0
<4> [350.600382] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.600387] process_one_work+0x239/0x760
<4> [350.600393] worker_thread+0x200/0x3f0
<4> [350.600396] ? __pfx_worker_thread+0x10/0x10
<4> [350.600398] kthread+0x10d/0x150
<4> [350.600401] ? __pfx_kthread+0x10/0x10
<4> [350.600405] ret_from_fork+0x3d4/0x480
<4> [350.600407] ? __pfx_kthread+0x10/0x10
<4> [350.600410] ret_from_fork_asm+0x1a/0x30
<4> [350.600417] </TASK>
<4> [350.600419] irq event stamp: 1890349
<4> [350.600420] hardirqs last enabled at (1890355): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.600423] hardirqs last disabled at (1890360): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.600425] softirqs last enabled at (1889600): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.600427] softirqs last disabled at (1889573): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.600429] ---[ end trace 0000000000000000 ]---
<6> [350.600431] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.600501] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.600509] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.600919] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.601121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.601219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.601312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.601398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.601485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.601572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.601661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.601751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.601833] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.601929] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.602046] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.603052] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.612064] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 8ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.612323] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [350.613357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [350.613437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [350.613513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [350.613590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [350.613666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [350.613740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [350.613814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [350.613888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [350.613961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [350.614033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [350.614107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [350.614195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [350.614280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [350.614356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [350.614431] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [350.614506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [350.614580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [350.614653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [350.614725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [350.614808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [350.614888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [350.614967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [350.615048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [350.615126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [350.615220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [350.615312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [350.615391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [350.615472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [350.615552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [350.615630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [350.615710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [350.615786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [350.615868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [350.615950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [350.616035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [350.616115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [350.616209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [350.616295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [350.616372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [350.616447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [350.616526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [350.616602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [350.616683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [350.616763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [350.616845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [350.616923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [350.617000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [350.617075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [350.617159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [350.617245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [350.617320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [350.617399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [350.617474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [350.617553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [350.617632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [350.617711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [350.617789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [350.617865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [350.617944] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [350.618062] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [350.618067] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278505, lrc_seqno=278505, guc_id=0, flags=0x73 in no process [-1]
<7> [350.618069] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.618128] ------------[ cut here ]------------
<4> [350.618129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.618130] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [350.618218] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.618303] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.618308] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.618311] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.618312] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.618313] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.618318] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.618388] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.618390] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.618392] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.618393] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.618394] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.618396] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.618397] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.618398] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [350.618400] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.618401] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [350.618403] PKRU: 55555554
<4> [350.618404] Call Trace:
<4> [350.618405] <TASK>
<4> [350.618409] ? lock_sync+0x100/0x100
<4> [350.618413] ? lock_release+0xd0/0x2b0
<4> [350.618418] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.618424] process_one_work+0x239/0x760
<4> [350.618430] worker_thread+0x200/0x3f0
<4> [350.618432] ? __pfx_worker_thread+0x10/0x10
<4> [350.618435] kthread+0x10d/0x150
<4> [350.618438] ? __pfx_kthread+0x10/0x10
<4> [350.618442] ret_from_fork+0x3d4/0x480
<4> [350.618443] ? __pfx_kthread+0x10/0x10
<4> [350.618447] ret_from_fork_asm+0x1a/0x30
<4> [350.618454] </TASK>
<4> [350.618455] irq event stamp: 1893479
<4> [350.618456] hardirqs last enabled at (1893485): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.618459] hardirqs last disabled at (1893490): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.618461] softirqs last enabled at (1892470): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.618464] softirqs last disabled at (1892465): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.618466] ---[ end trace 0000000000000000 ]---
<7> [350.619822] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [350.619929] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [350.620804] xe 0000:03:00.0: [drm:intel_hdmi_compute_clock [xe]] picking 12 bpc for HDMI output (pipe bpp: 36)
<7> [350.620917] xe 0000:03:00.0: [drm:intel_atomic_check [xe]] [CRTC:505:pipe D] hw max bpp: 36, pipe bpp: 36, dithering: 0
<7> [350.621029] xe 0000:03:00.0: [drm:intel_ddi_compute_config_late [xe]] [ENCODER:538:DDI TC3/PHY H] [CRTC:505:pipe D]
<7> [350.621135] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.enable (expected no, found yes)
<7> [350.621257] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.active (expected no, found yes)
<7> [350.621359] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in cpu_transcoder (expected -1, found 3)
<7> [350.621462] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in lane_count (expected 0, found 4)
<7> [350.621562] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in output_types (expected 0x00000000, found 0x00000040)
<7> [350.621660] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in framestart_delay (expected 0, found 1)
<7> [350.621754] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hdisplay (expected 0, found 720)
<7> [350.621849] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_htotal (expected 0, found 858)
<7> [350.621944] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hblank_start (expected 0, found 720)
<7> [350.622039] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hblank_end (expected 0, found 858)
<7> [350.622134] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hsync_start (expected 0, found 736)
<7> [350.622243] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hsync_end (expected 0, found 798)
<7> [350.622338] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vdisplay (expected 0, found 480)
<7> [350.622431] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vblank_start (expected 0, found 480)
<7> [350.622525] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vsync_start (expected 0, found 489)
<7> [350.622618] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vsync_end (expected 0, found 495)
<7> [350.622714] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vtotal (expected 0, found 525)
<7> [350.622812] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vblank_end (expected 0, found 525)
<7> [350.622910] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hdisplay (expected 0, found 720)
<7> [350.623007] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_htotal (expected 0, found 858)
<7> [350.623102] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hblank_start (expected 0, found 720)
<7> [350.623211] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hblank_end (expected 0, found 858)
<7> [350.623307] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hsync_start (expected 0, found 736)
<7> [350.623401] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hsync_end (expected 0, found 798)
<7> [350.623494] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vdisplay (expected 0, found 480)
<7> [350.623588] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vblank_start (expected 0, found 480)
<7> [350.623682] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vsync_start (expected 0, found 489)
<7> [350.623776] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vsync_end (expected 0, found 495)
<7> [350.623875] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vtotal (expected 0, found 525)
<7> [350.623974] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vblank_end (expected 0, found 525)
<7> [350.624070] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in pixel_multiplier (expected 0, found 1)
<7> [350.624170] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.flags (2) (expected 0, found 2)
<7> [350.624268] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.flags (8) (expected 0, found 8)
<7> [350.624363] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in has_hdmi_sink (expected no, found yes)
<7> [350.624457] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in has_infoframe (expected no, found yes)
<7> [350.624552] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in dpll_hw_state
<7> [350.624646] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] expected:
<7> [350.624740] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cx0pll_hw_state: lane_count: 4, ssc_enabled: no, use_c10: no, tbt_mode: no
<7> [350.624833] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] c20pll_hw_state:
<7> [350.624925] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] tx[0] = 0xbe88, tx[1] = 0x4800, tx[2] = 0x0000
<7> [350.625019] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cmn[0] = 0x0500, cmn[1] = 0x0005, cmn[2] = 0x0000, cmn[3] = 0x0000
<7> [350.625113] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[0] = 0x308c
<7> [350.625214] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[1] = 0x2110
<7> [350.625305] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[2] = 0xcc9c
<7> [350.625400] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[3] = 0xbfc1
<7> [350.625493] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[4] = 0x4b9a
<7> [350.625586] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[5] = 0x3f81
<7> [350.625678] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[6] = 0x2000
<7> [350.625770] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[7] = 0x0001
<7> [350.625863] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[8] = 0x5000
<7> [350.625954] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[9] = 0x0000
<7> [350.626051] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[10] = 0x0000
<7> [350.626152] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] vdr: custom width: 0x00, serdes rate: 0x42, hdmi rate: 0x00
<7> [350.626249] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] found:
<7> [350.626346] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cx0pll_hw_state: lane_count: 4, ssc_enabled: no, use_c10: no, tbt_mode: no
<7> [350.626441] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] c20pll_hw_state:
<7> [350.626536] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] tx[0] = 0xbe88, tx[1] = 0x5800, tx[2] = 0x0000
<7> [350.626631] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cmn[0] = 0x0500, cmn[1] = 0x0005, cmn[2] = 0x0000, cmn[3] = 0x0000
<7> [350.626726] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[0] = 0x80a8
<7> [350.626817] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[1] = 0x7d80
<7> [350.626910] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[2] = 0x8a06
<7> [350.627001] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[3] = 0x8f1c
<7> [350.627096] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[4] = 0x0000
<7> [350.627205] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[5] = 0x0000
<7> [350.627298] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[6] = 0x2200
<7> [350.627391] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[7] = 0xffff
<7> [350.627484] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[8] = 0x75dd
<7> [350.627575] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[9] = 0xdddd
<7> [350.627668] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[10] = 0x0001
<7> [350.627761] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] vdr: custom width: 0x00, serdes rate: 0x00, hdmi rate: 0x00
<7> [350.627855] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in pipe_bpp (expected 0, found 36)
<7> [350.627949] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_clock (expected 0, found 27027)
<7> [350.628043] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_clock (expected 0, found 27027)
<7> [350.628136] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in port_clock (expected 0, found 40541)
<7> [350.628247] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in infoframes.enable (expected 0x00000000, found 0x00000071)
<7> [350.628341] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in infoframes.gcp (expected 0x00000000, found 0x00000006)
<7> [350.628436] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in avi infoframe
<7> [350.628529] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] expected:
<7> [350.628621] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] found:
<7> [350.628714] xe 0000:03:00.0: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
<7> [350.628716] xe 0000:03:00.0: colorspace: RGB
<7> [350.628717] xe 0000:03:00.0: scan mode: Underscan
<7> [350.628719] xe 0000:03:00.0: colorimetry: No Data
<7> [350.628720] xe 0000:03:00.0: picture aspect: 16:9
<7> [350.628721] xe 0000:03:00.0: active aspect: Same as Picture
<7> [350.628722] xe 0000:03:00.0: itc: No Data
<7> [350.628723] xe 0000:03:00.0: extended colorimetry: xvYCC 601
<7> [350.628725] xe 0000:03:00.0: quantization range: Default
<7> [350.628726] xe 0000:03:00.0: nups: Unknown Non-uniform Scaling
<7> [350.628727] xe 0000:03:00.0: video code: 3
<7> [350.628728] xe 0000:03:00.0: ycc quantization range: Full
<7> [350.628729] xe 0000:03:00.0: hdmi content type: Graphics
<7> [350.628731] xe 0000:03:00.0: pixel repeat: 0
<7> [350.628732] xe 0000:03:00.0: bar top 0, bottom 0, left 0, right 0
<7> [350.628734] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in spd infoframe
<7> [350.628833] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] expected:
<7> [350.628933] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] found:
<7> [350.629030] xe 0000:03:00.0: HDMI infoframe: Source Product Description (SPD), version 1, length 25
<7> [350.629032] xe 0000:03:00.0: vendor: Intel
<7> [350.629033] xe 0000:03:00.0: product: Discrete gfx
<7> [350.629034] xe 0000:03:00.0: source device information: PC General (0x9)
<7> [350.629036] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hdmi infoframe
<7> [350.629132] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] expected:
<7> [350.629240] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] found:
<7> [350.629335] xe 0000:03:00.0: HDMI infoframe: Vendor, version 1, length 4
<7> [350.629336] xe 0000:03:00.0: empty frame
<7> [350.629338] xe 0000:03:00.0: [drm:intel_atomic_check [xe]] [CRTC:505:pipe D] fastset requirement not met, forcing full modeset
<7> [350.629467] xe 0000:03:00.0: [drm:skl_compute_wm [xe]] Enabled dbuf slices 0x1 -> 0x3 (total dbuf slices 0xf), mbus joined? no->no
<7> [350.629552] xe 0000:03:00.0: [drm:skl_compute_wm [xe]] [CRTC:505:pipe D] dbuf slices 0x0 -> 0x3, ddb (0 - 0) -> (0 - 2048), active pipes 0x0 -> 0x8
<7> [350.629639] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:389:plane 1D] ddb ( 0 - 0) -> ( 0 - 2016), size 0 -> 2016
<7> [350.629718] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:499:cursor D] ddb ( 0 - 0) -> (2016 - 2048), size 0 -> 32
<7> [350.629797] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [350.629874] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 4, 4, 5, 5, 8, 8, 8, 8, 0, 7, 0
<7> [350.629951] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 25, 25, 31, 31, 49, 49, 49, 49, 63, 43, 63
<7> [350.630040] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 49, 49, 73, 73, 73, 73, 0, 0, 64, 73, 73
<7> [350.630139] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:505:pipe D] data rate 108108 num active planes 1
<7> [350.630257] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 114 qgv_peak_bw: 48000
<7> [350.630357] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 114 qgv_peak_bw: 48000
<7> [350.630454] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 114
<7> [350.630566] xe 0000:03:00.0: [drm:intel_cdclk_atomic_check [xe]] [CRTC:505:pipe D] min cdclk: 0 kHz -> 192000 kHz
<7> [350.630680] xe 0000:03:00.0: [drm:intel_cdclk_update_dbuf_bw_min_cdclk [xe]] dbuf bandwidth min cdclk: 0 kHz -> 1690 kHz
<7> [350.630791] xe 0000:03:00.0: [drm:intel_find_dpll [xe]] [CRTC:505:pipe D] allocated TC PLL 3
<7> [350.630888] xe 0000:03:00.0: [drm:intel_dpll_crtc_get [xe]] [CRTC:505:pipe D] reserving TC PLL 3
<7> [350.631027] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [CRTC:505:pipe D] enable: yes [modeset]
<7> [350.631158] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] active: yes, output_types: HDMI (0x40), output format: RGB, sink format: RGB
<7> [350.631260] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] cpu_transcoder: D, pipe bpp: 36, dithering: 0
<7> [350.631360] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] MST master transcoder: <invalid>
<7> [350.631457] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] port sync: master transcoder: <invalid>, slave transcoder bitmask = 0x0
<7> [350.631552] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] joiner: no, pipes: 0x0
<7> [350.631646] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] splitter: disabled, link count 0, overlap 0
<7> [350.631739] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] audio: 1, infoframes: 1, infoframes enabled: 0x71
<7> [350.631832] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] GCP: 0x6
<7> [350.631926] xe 0000:03:00.0: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
<7> [350.631928] xe 0000:03:00.0: colorspace: RGB
<7> [350.631929] xe 0000:03:00.0: scan mode: Underscan
<7> [350.631930] xe 0000:03:00.0: colorimetry: No Data
<7> [350.631931] xe 0000:03:00.0: picture aspect: 16:9
<7> [350.631932] xe 0000:03:00.0: active aspect: Same as Picture
<7> [350.631934] xe 0000:03:00.0: itc: No Data
<7> [350.631935] xe 0000:03:00.0: extended colorimetry: xvYCC 601
<7> [350.631936] xe 0000:03:00.0: quantization range: Default
<7> [350.631937] xe 0000:03:00.0: nups: Unknown Non-uniform Scaling
<7> [350.631938] xe 0000:03:00.0: video code: 3
<7> [350.631940] xe 0000:03:00.0: ycc quantization range: Full
<7> [350.631941] xe 0000:03:00.0: hdmi content type: Graphics
<7> [350.631942] xe 0000:03:00.0: pixel repeat: 0
<7> [350.631943] xe 0000:03:00.0: bar top 0, bottom 0, left 0, right 0
<7> [350.631945] xe 0000:03:00.0: HDMI infoframe: Source Product Description (SPD), version 1, length 25
<7> [350.631946] xe 0000:03:00.0: vendor: Intel
<7> [350.631947] xe 0000:03:00.0: product: Discrete gfx
<7> [350.631949] xe 0000:03:00.0: source device information: PC General (0x9)
<7> [350.631950] xe 0000:03:00.0: HDMI infoframe: Vendor, version 1, length 4
<7> [350.631951] xe 0000:03:00.0: empty frame
<7> [350.631953] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] ELD: 10 00 07 00 66 10 00 01 00 00 00 00 00 00 00 00
<7> [350.632047] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] ELD: 05 e3 cd 0c 32 38 45 38 35 30 09 07 07 00 00 00
<7> [350.632148] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] scanline offset: 1
<7> [350.632241] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] framestart delay: 1, MSA timing delay: 0, set context latency: 0
<7> [350.632333] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vrr: no, fixed rr: yes, vmin: 525, vmax: 525, flipline: 525, pipeline full: 0, guardband: 45 vsync start: 36, vsync end: 30
<7> [350.632426] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vrr: vmin vblank: 480, vmax vblank: 480, vmin vtotal: 525, vmax vtotal: 525
<7> [350.632517] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vrr: dc balance: no, vmin: 0 vmax: 0 guardband: 0, slope: 0 max increase: 0 max decrease: 0 vblank target: 0
<7> [350.632609] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] requested mode: "720x480": 60 27027 720 736 798 858 480 489 495 525 0x40 0xa
<7> [350.632702] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] adjusted mode: "720x480": 60 27027 720 736 798 858 480 489 495 525 0x40 0xa
<7> [350.632796] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] crtc timings: clock=27027, hd=720 hb=720-858 hs=736-798 ht=858, vd=480 vb=480-525 vs=489-495 vt=525, flags=0xa
<7> [350.632890] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe mode: "720x480": 60 27027 720 736 798 858 480 489 495 525 0x40 0xa
<7> [350.632983] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] crtc timings: clock=27027, hd=720 hb=720-858 hs=736-798 ht=858, vd=480 vb=480-525 vs=489-495 vt=525, flags=0xa
<7> [350.633078] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] port clock: 40541, pixel rate 27027, min cdclk 192000, min voltage level 0
<7> [350.633178] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] linetime: 254, ips linetime: 0
<7> [350.633271] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] num_scalers: 2, scaler_users: 0x0, scaler_id: -1, scaling_filter: 0, sharpness_strength: 0
<7> [350.633365] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe src: 720x480+0+0
<7> [350.633457] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pch pfit: 0x0+0+0, disabled, force thru: no
<7> [350.633550] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] sharpness strength: 0, sharpness tap size: 0, sharpness enable: 0
<7> [350.633647] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] ips: 0, double wide: 0, drrs: 0
<7> [350.633746] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] cx0pll_hw_state: lane_count: 4, ssc_enabled: no, use_c10: no, tbt_mode: no
<7> [350.633843] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] c20pll_hw_state:
<7> [350.633938] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] tx[0] = 0xbe88, tx[1] = 0x5800, tx[2] = 0x0000
<7> [350.634033] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] cmn[0] = 0x0500, cmn[1] = 0x0005, cmn[2] = 0x0000, cmn[3] = 0x0000
<7> [350.634128] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[0] = 0x80a8
<7> [350.634236] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[1] = 0x7d80
<7> [350.634329] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[2] = 0x8a06
<7> [350.634422] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[3] = 0x8f1c
<7> [350.634522] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[4] = 0x0000
<7> [350.634613] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[5] = 0x0000
<7> [350.634703] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[6] = 0x2200
<7> [350.634792] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[7] = 0xffff
<7> [350.634882] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[8] = 0x75dd
<7> [350.634974] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[9] = 0xdddd
<7> [350.635067] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[10] = 0x0001
<7> [350.635225] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vdr: custom width: 0x00, serdes rate: 0x00, hdmi rate: 0x00
<7> [350.635315] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] csc_mode: 0x0 gamma_mode: 0x0 gamma_enable: 0 csc_enable: 0
<7> [350.635406] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pre csc lut: 0 entries, post csc lut: 0 entries
<7> [350.635495] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: pre offsets: 0x0000 0x0000 0x0000
<7> [350.635585] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.635676] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.635767] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.635860] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: post offsets: 0x0000 0x0000 0x0000
<7> [350.635951] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: pre offsets: 0x0000 0x0000 0x0000
<7> [350.636043] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.636135] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.636257] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.636350] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: post offsets: 0x0000 0x0000 0x0000
<7> [350.636443] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:389:plane 1D] fb: [FB:667] 720x480 format = XR24 little-endian (0x34325258) modifier = 0x100000000000009, visible: yes
<7> [350.636535] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] rotation: 0x1, scaler: -1, scaling_filter: 0
<7> [350.636627] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] src: 720.000000x480.000000+0.000000+0.000000 dst: 720x480+0+0
<7> [350.636718] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:419:plane 2D] fb: [NOFB], visible: no
<7> [350.636810] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:449:plane 3D] fb: [NOFB], visible: no
<7> [350.636902] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:479:plane 4D] fb: [NOFB], visible: no
<7> [350.636993] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:489:plane 5D] fb: [NOFB], visible: no
<7> [350.637084] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:499:cursor D] fb: [NOFB], visible: no
<7> [350.637593] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling always-on
<7> [350.637700] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling DC_off
<7> [350.638150] xe 0000:03:00.0: [drm:gen9_set_dc_state.part.0 [xe]] Setting DC state from 02 to 00
<7> [350.638634] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling PW_2
<7> [350.638948] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling PW_D
<7> [350.639070] xe 0000:03:00.0: [drm:dss_pipe_gating_enable_disable [xe]] DSS clock gating enabled for pipe_mask=0x8 (CLKGATE_DIS_DSSDSC=0x2a000000)
<7> [350.639399] xe 0000:03:00.0: [drm:intel_pmdemand_program_params [xe]] initiate pmdemand request values: (0x1e03061 0x28d02972)
<7> [350.639732] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:507:DDI TC1/PHY F]
<7> [350.639872] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:509:DP-MST A]
<7> [350.639979] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:510:DP-MST B]
<7> [350.640070] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:511:DP-MST C]
<7> [350.640168] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:512:DP-MST D]
<7> [350.640256] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:526:DDI TC2/PHY G]
<7> [350.640342] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:528:DP-MST A]
<7> [350.640427] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:529:DP-MST B]
<7> [350.640511] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:530:DP-MST C]
<7> [350.640597] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:531:DP-MST D]
<7> [350.640684] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:538:DDI TC3/PHY H]
<7> [350.640769] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:542:DDI TC4/PHY I]
<7> [350.640854] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:544:DP-MST A]
<7> [350.640941] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:545:DP-MST B]
<7> [350.641028] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:546:DP-MST C]
<7> [350.641111] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:547:DP-MST D]
<7> [350.641205] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:508:DP-1]
<7> [350.641502] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:521:HDMI-A-1]
<7> [350.641602] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:527:DP-2]
<7> [350.641691] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:535:HDMI-A-2]
<7> [350.641989] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:543:DP-3]
<7> [350.642080] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:551:HDMI-A-4]
<7> [350.642188] xe 0000:03:00.0: [drm:gen9_dbuf_slices_update [xe]] Updating dbuf slices to 0x3
<7> [350.642554] xe 0000:03:00.0: [drm:intel_dbuf_mdclk_cdclk_ratio_update [xe]] Updating dbuf ratio to 2 (mbus joined: no)
<7> [350.642655] xe 0000:03:00.0: [drm:intel_dpll_enable [xe]] enable TC PLL 3 (active 0x8, on? 0) for [CRTC:505:pipe D]
<7> [350.642752] xe 0000:03:00.0: [drm:intel_dpll_enable [xe]] enabling TC PLL 3
<7> [350.645322] xe 0000:03:00.0: [drm:intel_enable_transcoder [xe]] enabling pipe D
<7> [350.663702] xe 0000:03:00.0: [drm:intel_audio_codec_enable [xe]] [CONNECTOR:539:HDMI-A-3][ENCODER:538:DDI TC3/PHY H] Enable audio codec on [CRTC:505:pipe D], 32 bytes ELD
<7> [350.680313] xe 0000:03:00.0: [drm:audio_config_hdmi_pixel_clock [xe]] Configuring HDMI audio for pixel clock 27027 (0x00030000)
<7> [350.680422] xe 0000:03:00.0: [drm:hsw_audio_config_update [xe]] using automatic N
<7> [350.680858] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:539:HDMI-A-3]
<7> [350.681215] xe 0000:03:00.0: [drm:intel_modeset_verify_crtc [xe]] [CRTC:505:pipe D]
<7> [350.685238] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe D
<7> [350.685439] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe D
<7> [350.765410] xe 0000:03:00.0: [drm:intel_hdmi_compute_clock [xe]] picking 12 bpc for HDMI output (pipe bpp: 36)
<7> [350.766009] xe 0000:03:00.0: [drm:intel_atomic_check [xe]] [CRTC:505:pipe D] hw max bpp: 36, pipe bpp: 36, dithering: 0
<7> [350.766624] xe 0000:03:00.0: [drm:intel_ddi_compute_config_late [xe]] [ENCODER:538:DDI TC3/PHY H] [CRTC:505:pipe D]
<7> [350.767186] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hdisplay (expected 720, found 1920)
<7> [350.767723] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_htotal (expected 858, found 2200)
<7> [350.768283] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hblank_start (expected 720, found 1920)
<7> [350.768918] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hblank_end (expected 858, found 2200)
<7> [350.769595] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hsync_start (expected 736, found 2008)
<7> [350.770100] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_hsync_end (expected 798, found 2052)
<7> [350.770673] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vdisplay (expected 480, found 1080)
<7> [350.771202] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vblank_start (expected 480, found 1080)
<7> [350.771702] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vsync_start (expected 489, found 1083)
<7> [350.772250] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vsync_end (expected 495, found 1088)
<7> [350.772720] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vtotal (expected 525, found 1125)
<7> [350.772826] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_vblank_end (expected 525, found 1125)
<7> [350.772917] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hdisplay (expected 720, found 1920)
<7> [350.773008] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_htotal (expected 858, found 2200)
<7> [350.773097] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hblank_start (expected 720, found 1920)
<7> [350.773196] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hblank_end (expected 858, found 2200)
<7> [350.773287] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hsync_start (expected 736, found 2008)
<7> [350.773373] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_hsync_end (expected 798, found 2052)
<7> [350.773462] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vdisplay (expected 480, found 1080)
<7> [350.773555] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vblank_start (expected 480, found 1080)
<7> [350.773644] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vsync_start (expected 489, found 1083)
<7> [350.773731] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vsync_end (expected 495, found 1088)
<7> [350.773822] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vtotal (expected 525, found 1125)
<7> [350.773913] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_vblank_end (expected 525, found 1125)
<7> [350.774009] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.flags (1) (expected 0, found 1)
<7> [350.774097] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.flags (2) (expected 2, found 0)
<7> [350.774196] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in dpll_hw_state
<7> [350.774287] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] expected:
<7> [350.774374] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cx0pll_hw_state: lane_count: 4, ssc_enabled: no, use_c10: no, tbt_mode: no
<7> [350.774460] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] c20pll_hw_state:
<7> [350.774546] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] tx[0] = 0xbe88, tx[1] = 0x5800, tx[2] = 0x0000
<7> [350.774632] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cmn[0] = 0x0500, cmn[1] = 0x0005, cmn[2] = 0x0000, cmn[3] = 0x0000
<7> [350.774721] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[0] = 0x80a8
<7> [350.774813] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[1] = 0x7d80
<7> [350.774901] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[2] = 0x8a06
<7> [350.774987] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[3] = 0x8f1c
<7> [350.775074] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[4] = 0x0000
<7> [350.775170] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[5] = 0x0000
<7> [350.775270] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[6] = 0x2200
<7> [350.775358] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[7] = 0xffff
<7> [350.775446] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[8] = 0x75dd
<7> [350.775535] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[9] = 0xdddd
<7> [350.775622] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[10] = 0x0001
<7> [350.775710] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] vdr: custom width: 0x00, serdes rate: 0x00, hdmi rate: 0x00
<7> [350.775798] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] found:
<7> [350.775885] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cx0pll_hw_state: lane_count: 4, ssc_enabled: no, use_c10: no, tbt_mode: no
<7> [350.775973] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] c20pll_hw_state:
<7> [350.776068] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] tx[0] = 0xbe88, tx[1] = 0x5800, tx[2] = 0x0000
<7> [350.776164] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] cmn[0] = 0x0500, cmn[1] = 0x0005, cmn[2] = 0x0000, cmn[3] = 0x0000
<7> [350.776260] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[0] = 0x40e8
<7> [350.776345] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[1] = 0x7d20
<7> [350.776431] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[2] = 0x0a06
<7> [350.776518] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[3] = 0x8f1c
<7> [350.776603] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[4] = 0x0000
<7> [350.776687] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[5] = 0x0000
<7> [350.776770] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[6] = 0x2200
<7> [350.776854] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[7] = 0xffff
<7> [350.776937] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[8] = 0x0400
<7> [350.777022] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[9] = 0x0000
<7> [350.777107] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] mpllb[10] = 0x0001
<7> [350.777206] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] vdr: custom width: 0x00, serdes rate: 0x00, hdmi rate: 0x00
<7> [350.777297] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.pipe_mode.crtc_clock (expected 27027, found 148500)
<7> [350.777383] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in hw.adjusted_mode.crtc_clock (expected 27027, found 148500)
<7> [350.777468] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in port_clock (expected 40541, found 222750)
<7> [350.777553] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] [CRTC:505:pipe D] fastset requirement not met in avi infoframe
<7> [350.777642] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] expected:
<7> [350.777731] xe 0000:03:00.0: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
<7> [350.777733] xe 0000:03:00.0: colorspace: RGB
<7> [350.777734] xe 0000:03:00.0: scan mode: Underscan
<7> [350.777735] xe 0000:03:00.0: colorimetry: No Data
<7> [350.777737] xe 0000:03:00.0: picture aspect: 16:9
<7> [350.777738] xe 0000:03:00.0: active aspect: Same as Picture
<7> [350.777739] xe 0000:03:00.0: itc: No Data
<7> [350.777740] xe 0000:03:00.0: extended colorimetry: xvYCC 601
<7> [350.777741] xe 0000:03:00.0: quantization range: Default
<7> [350.777742] xe 0000:03:00.0: nups: Unknown Non-uniform Scaling
<7> [350.777743] xe 0000:03:00.0: video code: 3
<7> [350.777744] xe 0000:03:00.0: ycc quantization range: Full
<7> [350.777745] xe 0000:03:00.0: hdmi content type: Graphics
<7> [350.777746] xe 0000:03:00.0: pixel repeat: 0
<7> [350.777748] xe 0000:03:00.0: bar top 0, bottom 0, left 0, right 0
<7> [350.777749] xe 0000:03:00.0: [drm:intel_pipe_config_compare [xe]] found:
<7> [350.777836] xe 0000:03:00.0: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
<7> [350.777838] xe 0000:03:00.0: colorspace: RGB
<7> [350.777839] xe 0000:03:00.0: scan mode: Underscan
<7> [350.777840] xe 0000:03:00.0: colorimetry: No Data
<7> [350.777841] xe 0000:03:00.0: picture aspect: No Data
<7> [350.777842] xe 0000:03:00.0: active aspect: Same as Picture
<7> [350.777843] xe 0000:03:00.0: itc: No Data
<7> [350.777844] xe 0000:03:00.0: extended colorimetry: xvYCC 601
<7> [350.777845] xe 0000:03:00.0: quantization range: Full
<7> [350.777846] xe 0000:03:00.0: nups: Unknown Non-uniform Scaling
<7> [350.777847] xe 0000:03:00.0: video code: 0
<7> [350.777848] xe 0000:03:00.0: ycc quantization range: Full
<7> [350.777849] xe 0000:03:00.0: hdmi content type: Graphics
<7> [350.777850] xe 0000:03:00.0: pixel repeat: 0
<7> [350.777852] xe 0000:03:00.0: bar top 0, bottom 0, left 0, right 0
<7> [350.777853] xe 0000:03:00.0: [drm:intel_atomic_check [xe]] [CRTC:505:pipe D] fastset requirement not met, forcing full modeset
<7> [350.777941] xe 0000:03:00.0: [drm:intel_dpll_crtc_put [xe]] [CRTC:505:pipe D] releasing TC PLL 3
<7> [350.778072] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:389:plane 1D] ddb ( 0 - 2016) -> ( 0 - 1989), size 2016 -> 1989
<7> [350.778155] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:499:cursor D] ddb (2016 - 2048) -> (1989 - 2048), size 32 -> 59
<7> [350.778243] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [350.778316] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] lines 4, 4, 5, 5, 8, 8, 8, 8, 0, 7, 0 -> 4, 4, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [350.778388] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] blocks 25, 25, 31, 31, 49, 49, 49, 49, 63, 43, 63 -> 62, 62, 139, 139, 261, 261, 261, 261, 137, 230, 244
<7> [350.778457] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:389:plane 1D] min_ddb 49, 49, 73, 73, 73, 73, 0, 0, 64, 73, 73 -> 123, 123, 245, 245, 367, 367, 0, 0, 138, 306, 306
<7> [350.778543] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:505:pipe D] data rate 594000 num active planes 1
<7> [350.778638] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [350.778727] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [350.778817] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [350.778931] xe 0000:03:00.0: [drm:intel_cdclk_update_dbuf_bw_min_cdclk [xe]] dbuf bandwidth min cdclk: 1690 kHz -> 9282 kHz
<7> [350.779032] xe 0000:03:00.0: [drm:intel_find_dpll [xe]] [CRTC:505:pipe D] allocated TC PLL 3
<7> [350.779121] xe 0000:03:00.0: [drm:intel_dpll_crtc_get [xe]] [CRTC:505:pipe D] reserving TC PLL 3
<7> [350.779229] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [CRTC:505:pipe D] enable: yes [modeset]
<7> [350.779328] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] active: yes, output_types: HDMI (0x40), output format: RGB, sink format: RGB
<7> [350.779421] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] cpu_transcoder: D, pipe bpp: 36, dithering: 0
<7> [350.779511] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] MST master transcoder: <invalid>
<7> [350.779598] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] port sync: master transcoder: <invalid>, slave transcoder bitmask = 0x0
<7> [350.779686] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] joiner: no, pipes: 0x0
<7> [350.779773] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] splitter: disabled, link count 0, overlap 0
<7> [350.779867] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] audio: 1, infoframes: 1, infoframes enabled: 0x71
<7> [350.779956] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] GCP: 0x6
<7> [350.780045] xe 0000:03:00.0: HDMI infoframe: Auxiliary Video Information (AVI), version 2, length 13
<7> [350.780046] xe 0000:03:00.0: colorspace: RGB
<7> [350.780047] xe 0000:03:00.0: scan mode: Underscan
<7> [350.780048] xe 0000:03:00.0: colorimetry: No Data
<7> [350.780049] xe 0000:03:00.0: picture aspect: No Data
<7> [350.780051] xe 0000:03:00.0: active aspect: Same as Picture
<7> [350.780052] xe 0000:03:00.0: itc: No Data
<7> [350.780053] xe 0000:03:00.0: extended colorimetry: xvYCC 601
<7> [350.780054] xe 0000:03:00.0: quantization range: Full
<7> [350.780055] xe 0000:03:00.0: nups: Unknown Non-uniform Scaling
<7> [350.780056] xe 0000:03:00.0: video code: 0
<7> [350.780057] xe 0000:03:00.0: ycc quantization range: Full
<7> [350.780058] xe 0000:03:00.0: hdmi content type: Graphics
<7> [350.780059] xe 0000:03:00.0: pixel repeat: 0
<7> [350.780060] xe 0000:03:00.0: bar top 0, bottom 0, left 0, right 0
<7> [350.780062] xe 0000:03:00.0: HDMI infoframe: Source Product Description (SPD), version 1, length 25
<7> [350.780063] xe 0000:03:00.0: vendor: Intel
<7> [350.780064] xe 0000:03:00.0: product: Discrete gfx
<7> [350.780065] xe 0000:03:00.0: source device information: PC General (0x9)
<7> [350.780067] xe 0000:03:00.0: HDMI infoframe: Vendor, version 1, length 4
<7> [350.780068] xe 0000:03:00.0: empty frame
<7> [350.780069] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] ELD: 10 00 07 00 66 10 00 01 00 00 00 00 00 00 00 00
<7> [350.780163] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] ELD: 05 e3 cd 0c 32 38 45 38 35 30 09 07 07 00 00 00
<7> [350.780258] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] scanline offset: 1
<7> [350.780348] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] framestart delay: 1, MSA timing delay: 0, set context latency: 0
<7> [350.780448] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vrr: no, fixed rr: yes, vmin: 1125, vmax: 1125, flipline: 1125, pipeline full: 0, guardband: 45 vsync start: 42, vsync end: 37
<7> [350.780542] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vrr: vmin vblank: 1080, vmax vblank: 1080, vmin vtotal: 1125, vmax vtotal: 1125
<7> [350.780635] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vrr: dc balance: no, vmin: 0 vmax: 0 guardband: 0, slope: 0 max increase: 0 max decrease: 0 vblank target: 0
<7> [350.780728] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] requested mode: "1920x1080": 60 148500 1920 2008 2052 2200 1080 1083 1088 1125 0x48 0x9
<7> [350.780823] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] adjusted mode: "1920x1080": 60 148500 1920 2008 2052 2200 1080 1083 1088 1125 0x48 0x9
<7> [350.780918] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] crtc timings: clock=148500, hd=1920 hb=1920-2200 hs=2008-2052 ht=2200, vd=1080 vb=1080-1125 vs=1083-1088 vt=1125, flags=0x9
<7> [350.781012] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe mode: "1920x1080": 60 148500 1920 2008 2052 2200 1080 1083 1088 1125 0x40 0x9
<7> [350.781106] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] crtc timings: clock=148500, hd=1920 hb=1920-2200 hs=2008-2052 ht=2200, vd=1080 vb=1080-1125 vs=1083-1088 vt=1125, flags=0x9
<7> [350.781219] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] port clock: 222750, pixel rate 148500, min cdclk 192000, min voltage level 0
<7> [350.781313] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] linetime: 119, ips linetime: 0
<7> [350.781407] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] num_scalers: 2, scaler_users: 0x0, scaler_id: -1, scaling_filter: 0, sharpness_strength: 0
<7> [350.781509] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe src: 1920x1080+0+0
<7> [350.781655] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pch pfit: 0x0+0+0, disabled, force thru: no
<7> [350.781757] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] sharpness strength: 0, sharpness tap size: 0, sharpness enable: 0
<7> [350.781846] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] ips: 0, double wide: 0, drrs: 0
<7> [350.781936] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] cx0pll_hw_state: lane_count: 4, ssc_enabled: no, use_c10: no, tbt_mode: no
<7> [350.782025] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] c20pll_hw_state:
<7> [350.782111] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] tx[0] = 0xbe88, tx[1] = 0x5800, tx[2] = 0x0000
<7> [350.782215] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] cmn[0] = 0x0500, cmn[1] = 0x0005, cmn[2] = 0x0000, cmn[3] = 0x0000
<7> [350.782312] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[0] = 0x40e8
<7> [350.782401] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[1] = 0x7d20
<7> [350.782486] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[2] = 0x0a06
<7> [350.782570] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[3] = 0x8f1c
<7> [350.782655] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[4] = 0x0000
<7> [350.782739] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[5] = 0x0000
<7> [350.782823] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[6] = 0x2200
<7> [350.782907] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[7] = 0xffff
<7> [350.782991] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[8] = 0x0400
<7> [350.783075] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[9] = 0x0000
<7> [350.783165] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] mpllb[10] = 0x0001
<7> [350.783262] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] vdr: custom width: 0x00, serdes rate: 0x00, hdmi rate: 0x00
<7> [350.783358] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] csc_mode: 0x0 gamma_mode: 0x0 gamma_enable: 0 csc_enable: 0
<7> [350.783446] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pre csc lut: 0 entries, post csc lut: 0 entries
<7> [350.783533] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: pre offsets: 0x0000 0x0000 0x0000
<7> [350.783620] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.783705] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.783789] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.783870] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] output csc: post offsets: 0x0000 0x0000 0x0000
<7> [350.783955] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: pre offsets: 0x0000 0x0000 0x0000
<7> [350.784038] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.784122] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.784221] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: coefficients: 0x0000 0x0000 0x0000
<7> [350.784309] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] pipe csc: post offsets: 0x0000 0x0000 0x0000
<7> [350.784394] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:389:plane 1D] fb: [FB:666] 1920x1080 format = XR24 little-endian (0x34325258) modifier = 0x100000000000009, visible: yes
<7> [350.784479] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] rotation: 0x1, scaler: -1, scaling_filter: 0
<7> [350.784564] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] src: 1920.000000x1080.000000+0.000000+0.000000 dst: 1920x1080+0+0
<7> [350.784649] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:419:plane 2D] fb: [NOFB], visible: no
<7> [350.784733] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:449:plane 3D] fb: [NOFB], visible: no
<7> [350.784818] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:479:plane 4D] fb: [NOFB], visible: no
<7> [350.784903] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:489:plane 5D] fb: [NOFB], visible: no
<7> [350.784987] xe 0000:03:00.0: [drm:intel_crtc_state_dump [xe]] [PLANE:499:cursor D] fb: [NOFB], visible: no
<7> [350.785590] xe 0000:03:00.0: [drm:intel_audio_codec_disable [xe]] [CONNECTOR:539:HDMI-A-3][ENCODER:538:DDI TC3/PHY H] Disable audio codec on [CRTC:505:pipe D]
<7> [350.814111] xe 0000:03:00.0: [drm:intel_disable_transcoder [xe]] disabling pipe D
<7> [350.831404] xe 0000:03:00.0: [drm:intel_dpll_disable [xe]] disable TC PLL 3 (active 0x8, on? 1) for [CRTC:505:pipe D]
<7> [350.833871] xe 0000:03:00.0: [drm:intel_dpll_disable [xe]] disabling TC PLL 3
<7> [350.834664] xe 0000:03:00.0: [drm:intel_pmdemand_program_params [xe]] initiate pmdemand request values: (0x1e03061 0x28d0df72)
<7> [350.835374] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:507:DDI TC1/PHY F]
<7> [350.835862] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:509:DP-MST A]
<7> [350.836322] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:510:DP-MST B]
<7> [350.836731] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:511:DP-MST C]
<7> [350.837141] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:512:DP-MST D]
<7> [350.837576] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:526:DDI TC2/PHY G]
<7> [350.837927] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:528:DP-MST A]
<7> [350.838303] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:529:DP-MST B]
<7> [350.838621] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:530:DP-MST C]
<7> [350.838950] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:531:DP-MST D]
<7> [350.839310] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:538:DDI TC3/PHY H]
<7> [350.839639] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:542:DDI TC4/PHY I]
<7> [350.839934] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:544:DP-MST A]
<7> [350.840248] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:545:DP-MST B]
<7> [350.840519] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:546:DP-MST C]
<7> [350.840787] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:547:DP-MST D]
<7> [350.841286] xe 0000:03:00.0: [drm:intel_dpll_enable [xe]] enable TC PLL 3 (active 0x8, on? 0) for [CRTC:505:pipe D]
<7> [350.841565] xe 0000:03:00.0: [drm:intel_dpll_enable [xe]] enabling TC PLL 3
<7> [350.844173] xe 0000:03:00.0: [drm:intel_enable_transcoder [xe]] enabling pipe D
<7> [350.862065] xe 0000:03:00.0: [drm:intel_audio_codec_enable [xe]] [CONNECTOR:539:HDMI-A-3][ENCODER:538:DDI TC3/PHY H] Enable audio codec on [CRTC:505:pipe D], 32 bytes ELD
<7> [350.878661] xe 0000:03:00.0: [drm:audio_config_hdmi_pixel_clock [xe]] Configuring HDMI audio for pixel clock 148500 (0x00090000)
<7> [350.879243] xe 0000:03:00.0: [drm:hsw_audio_config_update [xe]] using automatic N
<7> [350.880203] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:539:HDMI-A-3]
<7> [350.880951] xe 0000:03:00.0: [drm:intel_modeset_verify_crtc [xe]] [CRTC:505:pipe D]
<7> [350.886675] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe D
<5> [350.981065] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278506, lrc_seqno=278506, guc_id=0, flags=0x73 in no process [-1]
<7> [350.981106] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [350.981589] ------------[ cut here ]------------
<4> [350.981596] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [350.981604] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:38/5132
<7> [350.981934] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:653]
<4> [350.981991] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [350.982350] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [350.982378] CPU: 8 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [350.982395] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [350.982402] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [350.982411] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [350.982439] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [350.982807] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [350.982817] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [350.982829] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [350.982838] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [350.982846] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [350.982853] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [350.982860] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [350.982868] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [350.982876] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [350.982884] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [350.982892] PKRU: 55555554
<4> [350.982899] Call Trace:
<4> [350.982907] <TASK>
<4> [350.982929] ? lock_sync+0x100/0x100
<4> [350.982955] ? lock_release+0xd0/0x2b0
<4> [350.982984] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [350.983012] process_one_work+0x239/0x760
<4> [350.983045] worker_thread+0x200/0x3f0
<4> [350.983059] ? __pfx_worker_thread+0x10/0x10
<4> [350.983071] kthread+0x10d/0x150
<4> [350.983086] ? __pfx_kthread+0x10/0x10
<4> [350.983105] ret_from_fork+0x3d4/0x480
<4> [350.983116] ? __pfx_kthread+0x10/0x10
<4> [350.983133] ret_from_fork_asm+0x1a/0x30
<4> [350.983186] </TASK>
<4> [350.983192] irq event stamp: 1895957
<4> [350.983198] hardirqs last enabled at (1895963): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [350.983214] hardirqs last disabled at (1895968): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [350.983225] softirqs last enabled at (1895186): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.983238] softirqs last disabled at (1895179): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [350.983250] ---[ end trace 0000000000000000 ]---
<6> [350.983260] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [350.983626] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [350.983680] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [350.983774] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [350.984439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [350.984885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [350.985390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [350.985826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [350.986307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [350.986752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [350.986874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [350.986961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [350.987047] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [350.987142] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [350.987269] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [350.988401] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [350.999197] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [350.999633] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.001181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.001404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.001613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.001806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.001998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.002202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.002389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.002569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.002743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.002919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.003092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.003292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.003459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.003623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.003780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.003936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.004093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.004273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.004426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.004592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.004757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.004914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.005071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.005244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.005387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.005529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.005671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.005812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.005950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.006091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.006250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.006385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.006525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.006661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.006793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.006918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.007044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.007172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.007295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.007407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.007528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.007642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.007807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.007946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.008068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.008201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.008307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.008412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.008523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.008626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.008728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.008830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.008929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.009027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.009125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.009237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.009331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.009424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.009519] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.009657] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.009662] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278506, lrc_seqno=278506, guc_id=0, flags=0x73 in no process [-1]
<7> [351.009665] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.009733] ------------[ cut here ]------------
<4> [351.009735] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.009736] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:40/5134
<4> [351.009819] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.009894] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.009899] CPU: 9 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.009903] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.009904] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.009906] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.009912] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.009994] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.009996] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [351.009999] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.010001] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.010002] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.010004] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.010005] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.010007] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [351.010009] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.010010] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.010012] PKRU: 55555554
<4> [351.010014] Call Trace:
<4> [351.010015] <TASK>
<4> [351.010020] ? lock_sync+0x100/0x100
<4> [351.010025] ? lock_release+0xd0/0x2b0
<4> [351.010031] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.010038] process_one_work+0x239/0x760
<4> [351.010045] worker_thread+0x200/0x3f0
<4> [351.010049] ? __pfx_worker_thread+0x10/0x10
<4> [351.010051] kthread+0x10d/0x150
<4> [351.010055] ? __pfx_kthread+0x10/0x10
<4> [351.010059] ret_from_fork+0x3d4/0x480
<4> [351.010061] ? __pfx_kthread+0x10/0x10
<4> [351.010065] ret_from_fork_asm+0x1a/0x30
<4> [351.010074] </TASK>
<4> [351.010075] irq event stamp: 2082511
<4> [351.010076] hardirqs last enabled at (2082517): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.010080] hardirqs last disabled at (2082522): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.010082] softirqs last enabled at (2081446): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.010085] softirqs last disabled at (2081439): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.010088] ---[ end trace 0000000000000000 ]---
<6> [351.010090] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.010183] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.010191] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.011506] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.011645] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.011765] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.011976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.012065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.012168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.012260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.012349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.012434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.012520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.012608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.012690] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.012785] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.012903] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.013926] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.024153] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.024405] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.025441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.025520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.025596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.025673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.025748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.025824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.025900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.025974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.026048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.026123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.026209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.026284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.026360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.026436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.026509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.026583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.026658] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.026730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.026804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.026885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.026964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.027041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.027119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.027211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.027289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.027366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.027445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.027527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.027608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.027687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.027765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.027842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.027925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.028009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.028092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.028179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.028258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.028334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.028410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.028486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.028564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.028640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.028716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.028793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.028877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.028956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.029034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.029111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.029199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.029274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.029349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.029429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.029509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.029587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.029663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.029741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.029815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.029888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.029966] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.030087] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.030091] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278506, lrc_seqno=278506, guc_id=0, flags=0x73 in no process [-1]
<7> [351.030094] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.030159] ------------[ cut here ]------------
<4> [351.030161] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.030162] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:38/5132
<4> [351.030235] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.030309] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.030314] CPU: 8 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.030318] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.030319] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.030321] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.030326] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.030398] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.030399] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.030402] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.030404] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.030405] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.030407] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.030408] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.030409] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [351.030411] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.030413] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [351.030415] PKRU: 55555554
<4> [351.030416] Call Trace:
<4> [351.030417] <TASK>
<4> [351.030421] ? lock_sync+0x100/0x100
<4> [351.030426] ? lock_release+0xd0/0x2b0
<4> [351.030431] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.030437] process_one_work+0x239/0x760
<4> [351.030443] worker_thread+0x200/0x3f0
<4> [351.030446] ? __pfx_worker_thread+0x10/0x10
<4> [351.030448] kthread+0x10d/0x150
<4> [351.030451] ? __pfx_kthread+0x10/0x10
<4> [351.030455] ret_from_fork+0x3d4/0x480
<4> [351.030457] ? __pfx_kthread+0x10/0x10
<4> [351.030460] ret_from_fork_asm+0x1a/0x30
<4> [351.030468] </TASK>
<4> [351.030469] irq event stamp: 1899687
<4> [351.030470] hardirqs last enabled at (1899693): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.030473] hardirqs last disabled at (1899698): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.030475] softirqs last enabled at (1898576): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.030477] softirqs last disabled at (1898569): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.030480] ---[ end trace 0000000000000000 ]---
<7> [351.031805] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.031912] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.049845] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278522, lrc_seqno=278522, guc_id=0, flags=0x73 in no process [-1]
<7> [351.049850] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.049918] ------------[ cut here ]------------
<4> [351.049919] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.049921] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:44/5138
<4> [351.049997] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.050063] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.050068] CPU: 0 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.050072] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.050073] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.050074] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.050080] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.050161] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.050164] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [351.050168] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.050171] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.050173] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.050176] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.050178] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.050180] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.050203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.050205] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.050208] PKRU: 55555554
<4> [351.050210] Call Trace:
<4> [351.050213] <TASK>
<4> [351.050223] ? lock_sync+0x100/0x100
<4> [351.050234] ? lock_release+0xd0/0x2b0
<4> [351.050245] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.050257] process_one_work+0x239/0x760
<4> [351.050272] worker_thread+0x200/0x3f0
<4> [351.050278] ? __pfx_worker_thread+0x10/0x10
<4> [351.050283] kthread+0x10d/0x150
<4> [351.050288] ? __pfx_kthread+0x10/0x10
<4> [351.050296] ret_from_fork+0x3d4/0x480
<4> [351.050300] ? __pfx_kthread+0x10/0x10
<4> [351.050307] ret_from_fork_asm+0x1a/0x30
<4> [351.050323] </TASK>
<4> [351.050326] irq event stamp: 2056247
<4> [351.050328] hardirqs last enabled at (2056253): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.050334] hardirqs last disabled at (2056258): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.050339] softirqs last enabled at (2056084): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.050344] softirqs last disabled at (2056079): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.050349] ---[ end trace 0000000000000000 ]---
<6> [351.050363] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.050435] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.050443] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.050847] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.051054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.051152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.051274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.051364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.051451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.051535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.051620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.051706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.051791] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.051888] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.052004] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.053026] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.063153] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.063405] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.064445] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.064528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.064608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.064689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.064768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.064849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.064930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.065010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.065086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.065168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.065251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.065326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.065399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.065472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.065547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.065623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.065697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.065772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.065849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.065931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.066013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.066092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.066182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.066270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.066347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.066425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.066503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.066582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.066662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.066744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.066827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.066909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.066994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.067079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.067172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.067265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.067347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.067425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.067502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.067579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.067662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.067741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.067818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.067895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.067976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.068052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.068129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.068224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.068314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.068390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.068466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.068544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.068619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.068695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.068770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.068851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.068931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.069009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.069088] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.069224] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.069229] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278522, lrc_seqno=278522, guc_id=0, flags=0x73 in no process [-1]
<7> [351.069231] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.069290] ------------[ cut here ]------------
<4> [351.069292] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.069293] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.069365] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.069430] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.069434] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.069437] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.069438] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.069440] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.069445] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.069516] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.069518] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.069520] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.069521] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.069523] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.069524] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.069525] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.069526] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.069528] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.069529] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.069531] PKRU: 55555554
<4> [351.069532] Call Trace:
<4> [351.069533] <TASK>
<4> [351.069537] ? lock_sync+0x100/0x100
<4> [351.069542] ? lock_release+0xd0/0x2b0
<4> [351.069547] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.069553] process_one_work+0x239/0x760
<4> [351.069559] worker_thread+0x200/0x3f0
<4> [351.069561] ? __pfx_worker_thread+0x10/0x10
<4> [351.069564] kthread+0x10d/0x150
<4> [351.069567] ? __pfx_kthread+0x10/0x10
<4> [351.069571] ret_from_fork+0x3d4/0x480
<4> [351.069573] ? __pfx_kthread+0x10/0x10
<4> [351.069576] ret_from_fork_asm+0x1a/0x30
<4> [351.069583] </TASK>
<4> [351.069584] irq event stamp: 1903343
<4> [351.069586] hardirqs last enabled at (1903349): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.069588] hardirqs last disabled at (1903354): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.069591] softirqs last enabled at (1902494): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.069593] softirqs last disabled at (1902487): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.069595] ---[ end trace 0000000000000000 ]---
<6> [351.069597] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.069668] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.069674] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.070083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.070301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.070389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.070481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.070568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.070655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.070742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.070828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.070918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.071003] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.071098] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.071228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.072248] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.082156] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.082406] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.083444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.083524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.083601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.083677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.083751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.083827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.083902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.083977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.084052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.084125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.084219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.084302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.084376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.084448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.084520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.084592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.084665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.084737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.084809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.084891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.084974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.085053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.085133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.085227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.085314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.085390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.085465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.085543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.085621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.085702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.085783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.085863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.085947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.086030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.086115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.086208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.086295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.086375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.086455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.086531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.086611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.086689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.086766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.086843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.086924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.087000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.087076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.087157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.087249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.087329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.087406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.087485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.087561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.087637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.087712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.087788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.087863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.087940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.088020] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.088135] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.088139] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278522, lrc_seqno=278522, guc_id=0, flags=0x73 in no process [-1]
<7> [351.088142] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.089078] ------------[ cut here ]------------
<4> [351.089080] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.089081] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.089169] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.089239] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.089244] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.089247] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.089248] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.089250] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.089255] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.089324] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.089326] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.089328] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.089329] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.089331] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.089332] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.089333] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.089334] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.089336] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.089337] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.089339] PKRU: 55555554
<4> [351.089340] Call Trace:
<4> [351.089341] <TASK>
<4> [351.089345] ? lock_sync+0x100/0x100
<4> [351.089349] ? lock_release+0xd0/0x2b0
<4> [351.089355] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.089360] process_one_work+0x239/0x760
<4> [351.089366] worker_thread+0x200/0x3f0
<4> [351.089369] ? __pfx_worker_thread+0x10/0x10
<4> [351.089371] kthread+0x10d/0x150
<4> [351.089374] ? __pfx_kthread+0x10/0x10
<4> [351.089378] ret_from_fork+0x3d4/0x480
<4> [351.089380] ? __pfx_kthread+0x10/0x10
<4> [351.089383] ret_from_fork_asm+0x1a/0x30
<4> [351.089390] </TASK>
<4> [351.089391] irq event stamp: 1906481
<4> [351.089393] hardirqs last enabled at (1906487): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.089395] hardirqs last disabled at (1906492): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.089397] softirqs last enabled at (1905732): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.089400] softirqs last disabled at (1905725): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.089402] ---[ end trace 0000000000000000 ]---
<7> [351.089434] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.089551] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.091245] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278523, lrc_seqno=278523, guc_id=0, flags=0x73 in no process [-1]
<7> [351.091277] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:669]
<7> [351.091249] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.091312] ------------[ cut here ]------------
<4> [351.091313] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.091315] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:38/5132
<4> [351.091391] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.091454] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.091458] CPU: 8 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.091461] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.091463] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.091464] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.091469] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.091541] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.091543] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.091545] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.091547] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.091548] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.091549] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.091550] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.091552] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [351.091553] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.091554] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [351.091556] PKRU: 55555554
<4> [351.091557] Call Trace:
<4> [351.091558] <TASK>
<4> [351.091562] ? lock_sync+0x100/0x100
<4> [351.091566] ? lock_release+0xd0/0x2b0
<4> [351.091572] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.091577] process_one_work+0x239/0x760
<4> [351.091583] worker_thread+0x200/0x3f0
<4> [351.091586] ? __pfx_worker_thread+0x10/0x10
<4> [351.091588] kthread+0x10d/0x150
<4> [351.091591] ? __pfx_kthread+0x10/0x10
<4> [351.091595] ret_from_fork+0x3d4/0x480
<4> [351.091597] ? __pfx_kthread+0x10/0x10
<4> [351.091600] ret_from_fork_asm+0x1a/0x30
<4> [351.091607] </TASK>
<4> [351.091608] irq event stamp: 1907467
<4> [351.091610] hardirqs last enabled at (1907473): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.091612] hardirqs last disabled at (1907478): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.091615] softirqs last enabled at (1905732): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.091617] softirqs last disabled at (1905725): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.091619] ---[ end trace 0000000000000000 ]---
<6> [351.091621] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.091693] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.091699] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.091892] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.092113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.092217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.092312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.092401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.092489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.092573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.092658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.092744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.092826] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.092922] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.093036] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.094183] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.104907] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.105160] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.106122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.106213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.106290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.106366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.106442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.106517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.106595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.106672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.106746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.106820] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.106893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.106964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.107037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.107109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.107209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.107282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.107354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.107430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.107507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.107589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.107670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.107748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.107827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.107904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.107981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.108057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.108134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.108226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.108306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.108386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.108469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.108549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.108634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.108717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.108800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.108881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.108960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.109036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.109112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.109197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.109276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.109352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.109429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.109503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.109583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.109665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.109745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.109822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.109901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.109976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.110052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.110130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.110220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.110300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.110379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.110459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.110534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.110612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.110691] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.110805] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.110809] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278523, lrc_seqno=278523, guc_id=0, flags=0x73 in no process [-1]
<7> [351.110812] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.110870] ------------[ cut here ]------------
<4> [351.110872] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.110873] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:44/5138
<4> [351.110945] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.111009] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.111013] CPU: 6 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.111016] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.111018] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.111019] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.111024] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.111096] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.111098] RSP: 0000:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [351.111100] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.111101] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.111103] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.111104] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.111105] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.111107] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [351.111108] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.111110] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [351.111111] PKRU: 55555554
<4> [351.111112] Call Trace:
<4> [351.111113] <TASK>
<4> [351.111118] ? lock_sync+0x100/0x100
<4> [351.111122] ? lock_release+0xd0/0x2b0
<4> [351.111128] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.111134] process_one_work+0x239/0x760
<4> [351.111140] worker_thread+0x200/0x3f0
<4> [351.111143] ? __pfx_worker_thread+0x10/0x10
<4> [351.111149] kthread+0x10d/0x150
<4> [351.111152] ? __pfx_kthread+0x10/0x10
<4> [351.111156] ret_from_fork+0x3d4/0x480
<4> [351.111158] ? __pfx_kthread+0x10/0x10
<4> [351.111162] ret_from_fork_asm+0x1a/0x30
<4> [351.111169] </TASK>
<4> [351.111171] irq event stamp: 2060205
<4> [351.111172] hardirqs last enabled at (2060211): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.111175] hardirqs last disabled at (2060216): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.111177] softirqs last enabled at (2059026): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.111179] softirqs last disabled at (2059017): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.111182] ---[ end trace 0000000000000000 ]---
<6> [351.111183] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.111254] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.111260] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.112499] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.112606] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.112712] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.112927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.113016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.113110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.113217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.113305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.113391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.113476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.113562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.113644] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.113737] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.113852] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.114862] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.125152] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.125402] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.126435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.126514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.126589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.126665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.126742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.126817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.126892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.126966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.127040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.127113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.127197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.127273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.127349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.127424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.127498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.127573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.127647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.127720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.127793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.127873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.127953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.128031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.128109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.128193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.128271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.128348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.128424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.128503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.128585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.128667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.128747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.128827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.128914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.129000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.129086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.129171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.129250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.129327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.129404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.129480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.129560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.129636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.129713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.129788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.129866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.129942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.130021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.130099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.130184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.130261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.130338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.130418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.130499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.130579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.130656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.130735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.130810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.130885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.130963] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.131081] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.131085] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278523, lrc_seqno=278523, guc_id=0, flags=0x73 in no process [-1]
<7> [351.131087] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.131173] ------------[ cut here ]------------
<4> [351.131176] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.131179] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:38/5132
<4> [351.131599] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.131733] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.131743] CPU: 9 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.131754] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.131756] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.131761] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.131778] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.132015] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.132017] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.132020] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.132021] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.132022] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.132024] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.132025] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.132026] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [351.132028] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.132029] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.132030] PKRU: 55555554
<4> [351.132032] Call Trace:
<4> [351.132033] <TASK>
<4> [351.132037] ? lock_sync+0x100/0x100
<4> [351.132042] ? lock_release+0xd0/0x2b0
<4> [351.132048] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.132054] process_one_work+0x239/0x760
<4> [351.132060] worker_thread+0x200/0x3f0
<4> [351.132063] ? __pfx_worker_thread+0x10/0x10
<4> [351.132066] kthread+0x10d/0x150
<4> [351.132069] ? __pfx_kthread+0x10/0x10
<4> [351.132072] ret_from_fork+0x3d4/0x480
<4> [351.132076] ? __pfx_kthread+0x10/0x10
<4> [351.132082] ret_from_fork_asm+0x1a/0x30
<4> [351.132095] </TASK>
<4> [351.132097] irq event stamp: 1911007
<4> [351.132099] hardirqs last enabled at (1911013): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.132103] hardirqs last disabled at (1911018): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.132106] softirqs last enabled at (1910080): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.132110] softirqs last disabled at (1910073): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.132113] ---[ end trace 0000000000000000 ]---
<7> [351.132092] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.132226] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.139403] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278526, lrc_seqno=278526, guc_id=0, flags=0x73 in no process [-1]
<7> [351.139407] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.139473] ------------[ cut here ]------------
<4> [351.139474] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.139476] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:40/5134
<4> [351.139551] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.139616] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.139621] CPU: 0 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.139623] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.139625] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.139626] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.139632] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.139705] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.139707] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [351.139709] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.139710] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.139712] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.139713] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.139714] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.139715] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.139717] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.139718] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.139720] PKRU: 55555554
<4> [351.139721] Call Trace:
<4> [351.139722] <TASK>
<4> [351.139726] ? lock_sync+0x100/0x100
<4> [351.139731] ? lock_release+0xd0/0x2b0
<4> [351.139737] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.139742] process_one_work+0x239/0x760
<4> [351.139748] worker_thread+0x200/0x3f0
<4> [351.139751] ? __pfx_worker_thread+0x10/0x10
<4> [351.139753] kthread+0x10d/0x150
<4> [351.139756] ? __pfx_kthread+0x10/0x10
<4> [351.139760] ret_from_fork+0x3d4/0x480
<4> [351.139762] ? __pfx_kthread+0x10/0x10
<4> [351.139765] ret_from_fork_asm+0x1a/0x30
<4> [351.139773] </TASK>
<4> [351.139774] irq event stamp: 2084245
<4> [351.139775] hardirqs last enabled at (2084251): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.139778] hardirqs last disabled at (2084256): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.139780] softirqs last enabled at (2083274): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [351.139783] softirqs last disabled at (2083272): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [351.139785] ---[ end trace 0000000000000000 ]---
<6> [351.139787] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.139859] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.139865] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.140155] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.140362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.140453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.140546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.140633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.140720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.140803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.140889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.140976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.141060] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.141164] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.141281] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.142290] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.153013] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.153263] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.154262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.154340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.154415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.154492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.154567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.154644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.154724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.154800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.154875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.154950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.155023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.155097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.155178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.155251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.155326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.155404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.155479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.155553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.155627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.155709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.155791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.155868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.155948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.156029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.156108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.156197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.156273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.156353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.156433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.156512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.156596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.156676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.156762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.156847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.156930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.157012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.157091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.157177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.157258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.157336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.157416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.157494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.157571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.157646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.157726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.157801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.157877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.157953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.158028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.158103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.158188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.158269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.158349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.158428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.158504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.158581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.158656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.158731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.158810] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.158927] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.158932] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278526, lrc_seqno=278526, guc_id=0, flags=0x73 in no process [-1]
<7> [351.158934] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.158992] ------------[ cut here ]------------
<4> [351.158993] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.158995] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:38/5132
<4> [351.159066] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.159128] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.159133] CPU: 9 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.159136] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.159137] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.159138] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.159143] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.159230] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.159232] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.159234] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.159236] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.159237] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.159239] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.159240] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.159242] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [351.159243] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.159245] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.159247] PKRU: 55555554
<4> [351.159248] Call Trace:
<4> [351.159249] <TASK>
<4> [351.159253] ? lock_sync+0x100/0x100
<4> [351.159258] ? lock_release+0xd0/0x2b0
<4> [351.159263] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.159269] process_one_work+0x239/0x760
<4> [351.159275] worker_thread+0x200/0x3f0
<4> [351.159278] ? __pfx_worker_thread+0x10/0x10
<4> [351.159280] kthread+0x10d/0x150
<4> [351.159283] ? __pfx_kthread+0x10/0x10
<4> [351.159287] ret_from_fork+0x3d4/0x480
<4> [351.159289] ? __pfx_kthread+0x10/0x10
<4> [351.159292] ret_from_fork_asm+0x1a/0x30
<4> [351.159300] </TASK>
<4> [351.159301] irq event stamp: 1914219
<4> [351.159302] hardirqs last enabled at (1914225): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.159305] hardirqs last disabled at (1914230): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.159307] softirqs last enabled at (1913308): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.159310] softirqs last disabled at (1913301): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.159312] ---[ end trace 0000000000000000 ]---
<6> [351.159314] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.159383] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.159389] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.160619] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.160722] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.160826] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.161032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.161123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.161242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.161336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.161425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.161511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.161595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.161681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.161762] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.161858] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.161973] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.162985] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.173154] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.173403] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.174370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.174461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.174541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.174619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.174694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.174770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.174844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.174918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.174991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.175065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.175138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.175237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.175320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.175397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.175473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.175549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.175624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.175697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.175770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.175850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.175930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.176010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.176092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.176180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.176270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.176346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.176424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.176507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.176587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.176665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.176744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.176823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.176906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.176989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.177072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.177162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.177256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.177337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.177417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.177494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.177572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.177649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.177727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.177803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.177882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.177959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.178037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.178116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.178208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.178297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.178374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.178453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.178530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.178605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.178682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.178761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.178838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.178914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.178993] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.179108] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.179112] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278526, lrc_seqno=278526, guc_id=0, flags=0x73 in no process [-1]
<7> [351.179114] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.179188] ------------[ cut here ]------------
<4> [351.179190] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.179193] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:40/5134
<4> [351.179276] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.179340] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.179345] CPU: 0 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.179348] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.179349] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.179351] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.179356] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.179426] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.179428] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [351.179430] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.179431] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.179433] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.179434] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.179435] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.179436] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.179438] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.179439] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.179441] PKRU: 55555554
<4> [351.179442] Call Trace:
<4> [351.179443] <TASK>
<4> [351.179447] ? lock_sync+0x100/0x100
<4> [351.179451] ? lock_release+0xd0/0x2b0
<4> [351.179457] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.179462] process_one_work+0x239/0x760
<4> [351.179468] worker_thread+0x200/0x3f0
<4> [351.179471] ? __pfx_worker_thread+0x10/0x10
<4> [351.179473] kthread+0x10d/0x150
<4> [351.179476] ? __pfx_kthread+0x10/0x10
<4> [351.179480] ret_from_fork+0x3d4/0x480
<4> [351.179482] ? __pfx_kthread+0x10/0x10
<4> [351.179485] ret_from_fork_asm+0x1a/0x30
<4> [351.179492] </TASK>
<4> [351.179494] irq event stamp: 2087653
<4> [351.179495] hardirqs last enabled at (2087659): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.179497] hardirqs last disabled at (2087664): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.179500] softirqs last enabled at (2086542): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.179502] softirqs last disabled at (2086535): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.179504] ---[ end trace 0000000000000000 ]---
<7> [351.180602] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.180729] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.180950] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278527, lrc_seqno=278527, guc_id=0, flags=0x73 in no process [-1]
<7> [351.180971] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:670]
<7> [351.180952] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.181012] ------------[ cut here ]------------
<4> [351.181013] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.181014] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.181085] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.181159] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.181167] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.181171] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.181173] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.181176] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.181184] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.181260] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.181262] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.181264] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.181266] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.181267] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.181268] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.181269] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.181271] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.181272] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.181273] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.181275] PKRU: 55555554
<4> [351.181276] Call Trace:
<4> [351.181277] <TASK>
<4> [351.181281] ? lock_sync+0x100/0x100
<4> [351.181285] ? lock_release+0xd0/0x2b0
<4> [351.181298] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.181304] process_one_work+0x239/0x760
<4> [351.181310] worker_thread+0x200/0x3f0
<4> [351.181313] ? __pfx_worker_thread+0x10/0x10
<4> [351.181315] kthread+0x10d/0x150
<4> [351.181318] ? __pfx_kthread+0x10/0x10
<4> [351.181322] ret_from_fork+0x3d4/0x480
<4> [351.181324] ? __pfx_kthread+0x10/0x10
<4> [351.181327] ret_from_fork_asm+0x1a/0x30
<4> [351.181334] </TASK>
<4> [351.181335] irq event stamp: 1915563
<4> [351.181337] hardirqs last enabled at (1915569): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.181339] hardirqs last disabled at (1915574): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.181342] softirqs last enabled at (1915310): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.181344] softirqs last disabled at (1915305): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.181346] ---[ end trace 0000000000000000 ]---
<6> [351.181348] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.181420] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.181426] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.181475] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.181730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.181822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.181916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.182004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.182092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.182186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.182288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.182376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.182456] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.182552] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.182668] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.183870] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.194514] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.194768] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.195794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.195876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.195953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.196032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.196111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.196200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.196277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.196352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.196426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.196500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.196572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.196645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.196718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.196791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.196863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.196936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.197010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.197086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.197168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.197252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.197334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.197413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.197492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.197569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.197646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.197727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.197806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.197886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.197965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.198044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.198121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.198209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.198293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.198376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.198458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.198538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.198621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.198699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.198777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.198853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.198933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.199010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.199086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.199166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.199246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.199322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.199397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.199471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.199547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.199622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.199698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.199780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.199859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.199937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.200012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.200089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.200169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.200245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.200324] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.200440] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.200444] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278527, lrc_seqno=278527, guc_id=0, flags=0x73 in no process [-1]
<7> [351.200447] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.200505] ------------[ cut here ]------------
<4> [351.200506] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.200508] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:40/5134
<4> [351.200579] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.200643] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.200647] CPU: 6 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.200650] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.200651] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.200653] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.200658] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.200728] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.200730] RSP: 0000:ffffc90004f23ca0 EFLAGS: 00010246
<4> [351.200732] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.200734] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.200735] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.200736] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.200737] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.200739] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [351.200740] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.200742] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [351.200743] PKRU: 55555554
<4> [351.200744] Call Trace:
<4> [351.200746] <TASK>
<4> [351.200750] ? lock_sync+0x100/0x100
<4> [351.200754] ? lock_release+0xd0/0x2b0
<4> [351.200760] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.200765] process_one_work+0x239/0x760
<4> [351.200771] worker_thread+0x200/0x3f0
<4> [351.200774] ? __pfx_worker_thread+0x10/0x10
<4> [351.200777] kthread+0x10d/0x150
<4> [351.200779] ? __pfx_kthread+0x10/0x10
<4> [351.200783] ret_from_fork+0x3d4/0x480
<4> [351.200785] ? __pfx_kthread+0x10/0x10
<4> [351.200788] ret_from_fork_asm+0x1a/0x30
<4> [351.200796] </TASK>
<4> [351.200797] irq event stamp: 2090787
<4> [351.200798] hardirqs last enabled at (2090793): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.200801] hardirqs last disabled at (2090798): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.200803] softirqs last enabled at (2089732): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.200805] softirqs last disabled at (2089727): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.200807] ---[ end trace 0000000000000000 ]---
<6> [351.200809] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.200880] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.200886] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.202142] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.202263] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.202370] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.202576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.202665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.202755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.202840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.202925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.203009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.203096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.203205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.203289] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.203383] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.203501] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.204525] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.215155] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.215404] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.216442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.216521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.216596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.216674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.216750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.216826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.216903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.216978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.217053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.217127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.217238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.217314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.217387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.217460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.217532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.217606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.217679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.217751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.217824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.217907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.217989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.218067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.218153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.218244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.218319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.218396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.218473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.218551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.218632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.218712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.218791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.218870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.218955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.219039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.219123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.219220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.219310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.219387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.219465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.219542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.219620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.219697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.219775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.219852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.219938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.220018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.220098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.220187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.220279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.220358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.220436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.220516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.220593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.220670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.220746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.220823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.220898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.220973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.221052] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.221175] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.221182] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278527, lrc_seqno=278527, guc_id=0, flags=0x73 in no process [-1]
<7> [351.221186] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.221252] ------------[ cut here ]------------
<4> [351.221253] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.221254] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.221328] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.221391] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.221396] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.221399] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.221400] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.221401] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.221407] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.221478] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.221479] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.221482] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.221483] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.221484] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.221486] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.221487] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.221488] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.221490] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.221491] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.221492] PKRU: 55555554
<4> [351.221494] Call Trace:
<4> [351.221495] <TASK>
<4> [351.221498] ? lock_sync+0x100/0x100
<4> [351.221503] ? lock_release+0xd0/0x2b0
<4> [351.221508] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.221514] process_one_work+0x239/0x760
<4> [351.221520] worker_thread+0x200/0x3f0
<4> [351.221523] ? __pfx_worker_thread+0x10/0x10
<4> [351.221525] kthread+0x10d/0x150
<4> [351.221528] ? __pfx_kthread+0x10/0x10
<4> [351.221532] ret_from_fork+0x3d4/0x480
<4> [351.221534] ? __pfx_kthread+0x10/0x10
<4> [351.221537] ret_from_fork_asm+0x1a/0x30
<4> [351.221545] </TASK>
<4> [351.221546] irq event stamp: 1919185
<4> [351.221547] hardirqs last enabled at (1919191): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.221550] hardirqs last disabled at (1919196): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.221552] softirqs last enabled at (1918168): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.221555] softirqs last disabled at (1918161): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.221557] ---[ end trace 0000000000000000 ]---
<7> [351.222907] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.223036] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.229877] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278530, lrc_seqno=278530, guc_id=0, flags=0x73 in no process [-1]
<7> [351.229880] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.229945] ------------[ cut here ]------------
<4> [351.229947] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.229949] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:38/5132
<4> [351.230024] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.230098] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.230104] CPU: 9 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.230107] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.230108] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.230110] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.230115] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.230197] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.230199] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.230201] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.230203] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.230204] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.230206] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.230207] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.230208] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [351.230210] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.230211] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.230213] PKRU: 55555554
<4> [351.230214] Call Trace:
<4> [351.230215] <TASK>
<4> [351.230219] ? lock_sync+0x100/0x100
<4> [351.230224] ? lock_release+0xd0/0x2b0
<4> [351.230229] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.230235] process_one_work+0x239/0x760
<4> [351.230241] worker_thread+0x200/0x3f0
<4> [351.230244] ? __pfx_worker_thread+0x10/0x10
<4> [351.230247] kthread+0x10d/0x150
<4> [351.230250] ? __pfx_kthread+0x10/0x10
<4> [351.230253] ret_from_fork+0x3d4/0x480
<4> [351.230256] ? __pfx_kthread+0x10/0x10
<4> [351.230259] ret_from_fork_asm+0x1a/0x30
<4> [351.230267] </TASK>
<4> [351.230268] irq event stamp: 1920111
<4> [351.230269] hardirqs last enabled at (1920117): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.230272] hardirqs last disabled at (1920122): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.230274] softirqs last enabled at (1919950): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.230277] softirqs last disabled at (1919943): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.230279] ---[ end trace 0000000000000000 ]---
<6> [351.230281] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.230352] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.230359] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.230768] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.230982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.231075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.231175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.231264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.231349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.231433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.231518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.231603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.231684] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.231778] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.231895] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.232907] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.243154] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.243406] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.244432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.244513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.244589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.244666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.244742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.244817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.244891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.244965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.245039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.245113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.245199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.245273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.245346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.245419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.245493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.245567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.245643] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.245720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.245798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.245879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.245960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.246038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.246118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.246203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.246282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.246360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.246437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.246516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.246595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.246672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.246750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.246827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.246910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.246995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.247082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.247184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.247266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.247344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.247422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.247498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.247577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.247654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.247730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.247805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.247885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.247962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.248038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.248114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.248208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.248283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.248360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.248442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.248521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.248599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.248674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.248750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.248825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.248900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.248982] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.249100] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.249105] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278530, lrc_seqno=278530, guc_id=0, flags=0x73 in no process [-1]
<7> [351.249107] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.249181] ------------[ cut here ]------------
<4> [351.249182] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.249184] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:26/3887
<4> [351.249256] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.249321] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.249326] CPU: 4 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.249329] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.249330] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.249332] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.249337] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.249408] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.249410] RSP: 0000:ffffc9000778fca0 EFLAGS: 00010246
<4> [351.249412] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.249414] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.249415] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.249416] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.249417] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.249419] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [351.249420] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.249422] CR2: 00007f796717f000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [351.249423] PKRU: 55555554
<4> [351.249424] Call Trace:
<4> [351.249425] <TASK>
<4> [351.249430] ? lock_sync+0x100/0x100
<4> [351.249434] ? lock_release+0xd0/0x2b0
<4> [351.249440] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.249445] process_one_work+0x239/0x760
<4> [351.249452] worker_thread+0x200/0x3f0
<4> [351.249454] ? __pfx_worker_thread+0x10/0x10
<4> [351.249457] kthread+0x10d/0x150
<4> [351.249460] ? __pfx_kthread+0x10/0x10
<4> [351.249463] ret_from_fork+0x3d4/0x480
<4> [351.249465] ? __pfx_kthread+0x10/0x10
<4> [351.249469] ret_from_fork_asm+0x1a/0x30
<4> [351.249476] </TASK>
<4> [351.249477] irq event stamp: 1453197
<4> [351.249478] hardirqs last enabled at (1453203): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.249481] hardirqs last disabled at (1453208): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.249483] softirqs last enabled at (1452274): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.249486] softirqs last disabled at (1452267): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.249488] ---[ end trace 0000000000000000 ]---
<6> [351.249490] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.249560] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.249566] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.250805] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.250909] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.251014] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.251237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.251331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.251428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.251516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.251602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.251687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.251772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.251859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.251941] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.252036] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.252173] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.253183] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.263918] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.264180] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.265274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.265355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.265431] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.265509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.265583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.265658] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.265733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.265807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.265879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.265953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.266026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.266099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.266182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.266257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.266331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.266405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.266479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.266555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.266632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.266717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.266799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.266878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.266958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.267034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.267112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.267197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.267275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.267358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.267440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.267521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.267600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.267678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.267763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.267846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.267930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.268010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.268090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.268178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.268258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.268336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.268417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.268494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.268572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.268650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.268730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.268807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.268888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.268966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.269044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.269118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.269204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.269283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.269360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.269437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.269518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.269599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.269677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.269753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.269832] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.269947] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.269951] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278530, lrc_seqno=278530, guc_id=0, flags=0x73 in no process [-1]
<7> [351.269954] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.270013] ------------[ cut here ]------------
<4> [351.270014] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.270016] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:40/5134
<4> [351.270087] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.270156] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.270161] CPU: 6 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.270164] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.270166] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.270167] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.270172] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.270242] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.270244] RSP: 0000:ffffc90004f23ca0 EFLAGS: 00010246
<4> [351.270246] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.270248] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.270249] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.270250] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.270251] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.270253] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [351.270254] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.270256] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [351.270257] PKRU: 55555554
<4> [351.270258] Call Trace:
<4> [351.270259] <TASK>
<4> [351.270264] ? lock_sync+0x100/0x100
<4> [351.270268] ? lock_release+0xd0/0x2b0
<4> [351.270274] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.270279] process_one_work+0x239/0x760
<4> [351.270285] worker_thread+0x200/0x3f0
<4> [351.270288] ? __pfx_worker_thread+0x10/0x10
<4> [351.270290] kthread+0x10d/0x150
<4> [351.270293] ? __pfx_kthread+0x10/0x10
<4> [351.270297] ret_from_fork+0x3d4/0x480
<4> [351.270299] ? __pfx_kthread+0x10/0x10
<4> [351.270302] ret_from_fork_asm+0x1a/0x30
<4> [351.270309] </TASK>
<4> [351.270310] irq event stamp: 2094233
<4> [351.270312] hardirqs last enabled at (2094239): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.270315] hardirqs last disabled at (2094244): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.270317] softirqs last enabled at (2093232): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.270319] softirqs last disabled at (2093227): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.270321] ---[ end trace 0000000000000000 ]---
<7> [351.271342] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.271446] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.273000] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278531, lrc_seqno=278531, guc_id=0, flags=0x73 in no process [-1]
<7> [351.273003] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.273062] ------------[ cut here ]------------
<4> [351.273064] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.273065] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:40/5134
<4> [351.273135] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.273206] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.273211] CPU: 6 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.273213] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.273214] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.273216] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [351.273244] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:671]
<4> [351.273221] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.273289] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.273291] RSP: 0000:ffffc90004f23ca0 EFLAGS: 00010246
<4> [351.273293] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.273294] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.273296] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.273297] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.273298] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.273299] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [351.273301] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.273302] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [351.273304] PKRU: 55555554
<4> [351.273305] Call Trace:
<4> [351.273306] <TASK>
<4> [351.273309] ? lock_sync+0x100/0x100
<4> [351.273313] ? lock_release+0xd0/0x2b0
<4> [351.273319] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.273324] process_one_work+0x239/0x760
<4> [351.273330] worker_thread+0x200/0x3f0
<4> [351.273333] ? __pfx_worker_thread+0x10/0x10
<4> [351.273336] kthread+0x10d/0x150
<4> [351.273338] ? __pfx_kthread+0x10/0x10
<4> [351.273342] ret_from_fork+0x3d4/0x480
<4> [351.273344] ? __pfx_kthread+0x10/0x10
<4> [351.273347] ret_from_fork_asm+0x1a/0x30
<4> [351.273354] </TASK>
<4> [351.273355] irq event stamp: 2095107
<4> [351.273357] hardirqs last enabled at (2095113): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.273359] hardirqs last disabled at (2095118): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.273361] softirqs last enabled at (2094438): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.273364] softirqs last disabled at (2094431): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.273366] ---[ end trace 0000000000000000 ]---
<6> [351.273368] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.273438] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.273444] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.273567] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.273818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.273912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.274014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.274110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.274253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.274358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.274448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.274540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.274625] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.274722] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.274836] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.276027] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.286156] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.286421] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.287569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.287651] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.287728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.287807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.287884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.287960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.288035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.288108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.288196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.288271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.288344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.288417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.288490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.288562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.288634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.288706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.288778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.288851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.288925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.289006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.289088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.289175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.289255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.289332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.289411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.289492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.289569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.289647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.289725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.289801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.289877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.289957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.290041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.290122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.290214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.290294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.290371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.290447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.290528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.290606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.290686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.290765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.290841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.290917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.290997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.291073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.291153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.291228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.291310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.291391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.291469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.291548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.291624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.291700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.291774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.291850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.291924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.291998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.292077] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.292196] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.292200] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278531, lrc_seqno=278531, guc_id=0, flags=0x73 in no process [-1]
<7> [351.292203] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.292261] ------------[ cut here ]------------
<4> [351.292262] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.292264] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:26/3887
<4> [351.292334] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.292398] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.292403] CPU: 6 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.292406] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.292407] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.292408] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.292413] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.292484] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.292486] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [351.292488] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.292490] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.292491] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.292492] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.292493] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.292495] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [351.292496] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.292497] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [351.292499] PKRU: 55555554
<4> [351.292500] Call Trace:
<4> [351.292501] <TASK>
<4> [351.292505] ? lock_sync+0x100/0x100
<4> [351.292510] ? lock_release+0xd0/0x2b0
<4> [351.292515] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.292520] process_one_work+0x239/0x760
<4> [351.292526] worker_thread+0x200/0x3f0
<4> [351.292529] ? __pfx_worker_thread+0x10/0x10
<4> [351.292531] kthread+0x10d/0x150
<4> [351.292534] ? __pfx_kthread+0x10/0x10
<4> [351.292538] ret_from_fork+0x3d4/0x480
<4> [351.292540] ? __pfx_kthread+0x10/0x10
<4> [351.292543] ret_from_fork_asm+0x1a/0x30
<4> [351.292550] </TASK>
<4> [351.292552] irq event stamp: 1456857
<4> [351.292553] hardirqs last enabled at (1456863): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.292555] hardirqs last disabled at (1456868): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.292558] softirqs last enabled at (1455630): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.292560] softirqs last disabled at (1455621): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.292562] ---[ end trace 0000000000000000 ]---
<6> [351.292564] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.292635] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.292641] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.293883] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.293988] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.294093] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.294303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.294393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.294487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.294577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.294666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.294751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.294837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.294922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.295003] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.295097] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.295235] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.296240] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.306962] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.307215] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.308243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.308322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.308397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.308473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.308547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.308622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.308695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.308769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.308843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.308918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.308994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.309070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.309150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.309225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.309300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.309372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.309445] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.309519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.309595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.309677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.309758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.309835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.309914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.309990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.310066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.310142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.310232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.310310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.310389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.310466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.310544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.310622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.310704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.310788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.310874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.310958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.311039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.311116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.311202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.311277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.311356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.311434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.311510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.311585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.311667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.311747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.311824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.311901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.311979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.312055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.312131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.312219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.312296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.312372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.312447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.312523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.312599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.312677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.312758] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.312875] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.312879] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278531, lrc_seqno=278531, guc_id=0, flags=0x73 in no process [-1]
<7> [351.312881] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.312939] ------------[ cut here ]------------
<4> [351.312941] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.312942] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:38/5132
<4> [351.313013] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.313077] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.313082] CPU: 9 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.313085] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.313086] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.313088] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.313093] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.313167] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.313169] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.313172] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.313173] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.313175] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.313176] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.313178] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.313179] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [351.313181] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.313183] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.313184] PKRU: 55555554
<4> [351.313185] Call Trace:
<4> [351.313186] <TASK>
<4> [351.313191] ? lock_sync+0x100/0x100
<4> [351.313195] ? lock_release+0xd0/0x2b0
<4> [351.313201] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.313206] process_one_work+0x239/0x760
<4> [351.313212] worker_thread+0x200/0x3f0
<4> [351.313215] ? __pfx_worker_thread+0x10/0x10
<4> [351.313218] kthread+0x10d/0x150
<4> [351.313221] ? __pfx_kthread+0x10/0x10
<4> [351.313224] ret_from_fork+0x3d4/0x480
<4> [351.313226] ? __pfx_kthread+0x10/0x10
<4> [351.313230] ret_from_fork_asm+0x1a/0x30
<4> [351.313238] </TASK>
<4> [351.313239] irq event stamp: 1923629
<4> [351.313240] hardirqs last enabled at (1923635): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.313243] hardirqs last disabled at (1923640): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.313245] softirqs last enabled at (1922536): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.313248] softirqs last disabled at (1922527): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.313250] ---[ end trace 0000000000000000 ]---
<7> [351.314542] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.314649] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.315517] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278547, lrc_seqno=278547, guc_id=0, flags=0x73 in no process [-1]
<7> [351.315520] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.315582] ------------[ cut here ]------------
<4> [351.315583] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.315585] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:26/3887
<4> [351.315657] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.315720] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.315733] CPU: 0 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.315736] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.315737] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.315739] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.315744] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.315827] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.315829] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [351.315831] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.315832] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.315834] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.315835] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.315836] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.315837] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.315839] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.315840] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.315842] PKRU: 55555554
<4> [351.315843] Call Trace:
<4> [351.315844] <TASK>
<4> [351.315848] ? lock_sync+0x100/0x100
<4> [351.315853] ? lock_release+0xd0/0x2b0
<4> [351.315858] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.315864] process_one_work+0x239/0x760
<4> [351.315870] worker_thread+0x200/0x3f0
<4> [351.315872] ? __pfx_worker_thread+0x10/0x10
<4> [351.315875] kthread+0x10d/0x150
<4> [351.315878] ? __pfx_kthread+0x10/0x10
<4> [351.315881] ret_from_fork+0x3d4/0x480
<4> [351.315883] ? __pfx_kthread+0x10/0x10
<4> [351.315887] ret_from_fork_asm+0x1a/0x30
<4> [351.315894] </TASK>
<4> [351.315895] irq event stamp: 1458603
<4> [351.315896] hardirqs last enabled at (1458609): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.315899] hardirqs last disabled at (1458614): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.315901] softirqs last enabled at (1456976): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [351.315904] softirqs last disabled at (1456974): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [351.315906] ---[ end trace 0000000000000000 ]---
<6> [351.315908] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.315978] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.315984] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.316031] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.316255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.316343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.316435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.316522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.316618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.316703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.316803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.316889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.316973] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.317070] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.317190] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.318267] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.328153] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.328403] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.329429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.329509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.329584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.329659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.329734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.329807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.329882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.329957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.330030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.330101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.330181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.330255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.330328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.330403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.330479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.330554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.330628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.330702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.330776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.330858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.330938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.331020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.331101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.331196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.331274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.331351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.331428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.331508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.331588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.331669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.331748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.331826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.331910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.331992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.332074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.332160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.332243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.332322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.332400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.332475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.332554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.332635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.332715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.332794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.332875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.332951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.333027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.333102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.333186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.333263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.333338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.333414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.333490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.333566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.333641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.333717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.333791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.333869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.333950] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.334066] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.334070] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278547, lrc_seqno=278547, guc_id=0, flags=0x73 in no process [-1]
<7> [351.334073] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.334131] ------------[ cut here ]------------
<4> [351.334132] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.334133] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#9: kworker/u64:38/5132
<4> [351.334216] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.334281] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.334286] CPU: 9 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.334289] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.334290] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.334291] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.334296] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.334367] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.334368] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.334371] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.334372] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.334373] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.334374] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.334376] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.334377] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000
<4> [351.334379] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.334380] CR2: 00007f79671cd000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.334381] PKRU: 55555554
<4> [351.334383] Call Trace:
<4> [351.334384] <TASK>
<4> [351.334388] ? lock_sync+0x100/0x100
<4> [351.334392] ? lock_release+0xd0/0x2b0
<4> [351.334397] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.334403] process_one_work+0x239/0x760
<4> [351.334409] worker_thread+0x200/0x3f0
<4> [351.334411] ? __pfx_worker_thread+0x10/0x10
<4> [351.334414] kthread+0x10d/0x150
<4> [351.334417] ? __pfx_kthread+0x10/0x10
<4> [351.334420] ret_from_fork+0x3d4/0x480
<4> [351.334422] ? __pfx_kthread+0x10/0x10
<4> [351.334426] ret_from_fork_asm+0x1a/0x30
<4> [351.334433] </TASK>
<4> [351.334434] irq event stamp: 1927103
<4> [351.334435] hardirqs last enabled at (1927109): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.334438] hardirqs last disabled at (1927114): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.334440] softirqs last enabled at (1926086): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.334442] softirqs last disabled at (1926079): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.334445] ---[ end trace 0000000000000000 ]---
<6> [351.334446] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.334517] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.334525] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.335774] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.335879] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [351.335985] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.336198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.336289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.336384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.336473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.336559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.336644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.336728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.336813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.336895] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.336989] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.337123] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.338137] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.348155] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.348408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.349380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.349460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.349536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.349612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.349688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.349763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.349840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.349918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.349993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.350067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.350142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.350231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.350305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.350378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.350451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.350524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.350596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.350668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.350740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.350820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.350900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.350979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.351060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.351138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.351231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.351307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.351382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.351460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.351542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.351623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.351703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.351783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.351867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.351950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.352034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.352114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.352204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.352281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.352358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.352435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.352518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.352598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.352676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.352751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.352832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.352908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.352983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.353058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.353134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.353223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.353298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.353376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.353453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.353534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.353612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.353703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.353803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.353888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.353968] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.354089] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.354093] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278547, lrc_seqno=278547, guc_id=0, flags=0x73 in no process [-1]
<7> [351.354096] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.354161] ------------[ cut here ]------------
<4> [351.354163] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.354164] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:44/5138
<4> [351.354237] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.354300] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.354305] CPU: 8 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.354308] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.354309] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.354311] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.354316] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.354386] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.354387] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [351.354390] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.354391] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.354392] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.354393] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.354395] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.354396] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [351.354398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.354399] CR2: 00007f79671c7000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [351.354400] PKRU: 55555554
<4> [351.354402] Call Trace:
<4> [351.354403] <TASK>
<4> [351.354407] ? lock_sync+0x100/0x100
<4> [351.354412] ? lock_release+0xd0/0x2b0
<4> [351.354417] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.354423] process_one_work+0x239/0x760
<4> [351.354429] worker_thread+0x200/0x3f0
<4> [351.354431] ? __pfx_worker_thread+0x10/0x10
<4> [351.354434] kthread+0x10d/0x150
<4> [351.354437] ? __pfx_kthread+0x10/0x10
<4> [351.354440] ret_from_fork+0x3d4/0x480
<4> [351.354442] ? __pfx_kthread+0x10/0x10
<4> [351.354446] ret_from_fork_asm+0x1a/0x30
<4> [351.354453] </TASK>
<4> [351.354454] irq event stamp: 2063711
<4> [351.354455] hardirqs last enabled at (2063717): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.354458] hardirqs last disabled at (2063722): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.354461] softirqs last enabled at (2062794): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.354463] softirqs last disabled at (2062787): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.354465] ---[ end trace 0000000000000000 ]---
<7> [351.355776] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.355882] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.948025] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278555, lrc_seqno=278555, guc_id=0, flags=0x73 in no process [-1]
<7> [351.948030] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.948139] ------------[ cut here ]------------
<4> [351.948140] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.948142] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.948253] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.948332] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.948337] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.948340] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.948341] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.948343] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.948348] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.948417] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.948419] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.948422] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.948423] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.948425] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.948426] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.948427] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.948429] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.948430] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.948432] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.948433] PKRU: 55555554
<4> [351.948435] Call Trace:
<4> [351.948436] <TASK>
<4> [351.948440] ? lock_sync+0x100/0x100
<4> [351.948444] ? lock_release+0xd0/0x2b0
<4> [351.948449] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.948455] process_one_work+0x239/0x760
<4> [351.948461] worker_thread+0x200/0x3f0
<4> [351.948464] ? __pfx_worker_thread+0x10/0x10
<4> [351.948466] kthread+0x10d/0x150
<4> [351.948469] ? __pfx_kthread+0x10/0x10
<4> [351.948473] ret_from_fork+0x3d4/0x480
<4> [351.948475] ? __pfx_kthread+0x10/0x10
<4> [351.948479] ret_from_fork_asm+0x1a/0x30
<4> [351.948486] </TASK>
<4> [351.948487] irq event stamp: 1930225
<4> [351.948488] hardirqs last enabled at (1930231): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.948491] hardirqs last disabled at (1930236): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.948494] softirqs last enabled at (1929476): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.948496] softirqs last disabled at (1929463): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.948499] ---[ end trace 0000000000000000 ]---
<6> [351.948501] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.948566] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.948572] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.948955] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.949196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.949316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.949411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.949498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.949579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.949663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.949745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.949825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.949901] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.949989] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.950114] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.951270] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.962009] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.962271] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.963246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.963319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.963389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.963459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.963528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.963598] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.963666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.963735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.963803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.963870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.963937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.964005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.964073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.964143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.964238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.964328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.964401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.964474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.964546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.964627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.964708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.964786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.964865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.964945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.965025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.965103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.965194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.965285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.965364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.965441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.965519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.965596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.965679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.965767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.965855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.965937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.966018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.966096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.966182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.966269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.966348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.966424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.966502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.966582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.966666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.966745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.966823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.966899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.966977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.967054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.967129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.967228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.967320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.967403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.967486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.967569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.967651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.967734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.967820] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.967946] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.967951] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278555, lrc_seqno=278555, guc_id=0, flags=0x73 in no process [-1]
<7> [351.967954] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.968018] ------------[ cut here ]------------
<4> [351.968020] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.968021] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.968099] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.968203] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.968208] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.968212] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.968213] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.968215] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.968221] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.968298] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.968300] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.968302] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.968304] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.968306] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.968307] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.968308] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.968310] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.968311] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.968313] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.968315] PKRU: 55555554
<4> [351.968316] Call Trace:
<4> [351.968317] <TASK>
<4> [351.968322] ? lock_sync+0x100/0x100
<4> [351.968327] ? lock_release+0xd0/0x2b0
<4> [351.968334] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.968340] process_one_work+0x239/0x760
<4> [351.968347] worker_thread+0x200/0x3f0
<4> [351.968351] ? __pfx_worker_thread+0x10/0x10
<4> [351.968353] kthread+0x10d/0x150
<4> [351.968357] ? __pfx_kthread+0x10/0x10
<4> [351.968361] ret_from_fork+0x3d4/0x480
<4> [351.968363] ? __pfx_kthread+0x10/0x10
<4> [351.968367] ret_from_fork_asm+0x1a/0x30
<4> [351.968376] </TASK>
<4> [351.968377] irq event stamp: 1933353
<4> [351.968378] hardirqs last enabled at (1933359): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.968382] hardirqs last disabled at (1933364): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.968384] softirqs last enabled at (1932448): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.968387] softirqs last disabled at (1932437): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.968390] ---[ end trace 0000000000000000 ]---
<6> [351.968392] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.968471] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.968478] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.968884] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.969103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.969225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.969366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.969476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.969585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.969691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.969798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.969907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.970011] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.970132] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.970302] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.971363] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [351.981158] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [351.981407] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [351.982581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [351.982660] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [351.982730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [351.982800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [351.982869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [351.982937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [351.983007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [351.983078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [351.983156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [351.983247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [351.983321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [351.983395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [351.983468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [351.983540] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [351.983614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [351.983691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [351.983767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [351.983842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [351.983915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [351.983994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [351.984075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [351.984163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [351.984256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [351.984334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [351.984410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [351.984485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [351.984561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [351.984641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [351.984721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [351.984799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [351.984877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [351.984954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [351.985039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [351.985122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [351.985216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [351.985308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [351.985391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [351.985470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [351.985548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [351.985624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [351.985703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [351.985780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [351.985855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [351.985932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [351.986015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [351.986094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [351.986181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [351.986268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [351.986346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [351.986421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [351.986497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [351.986575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [351.986653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [351.986735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [351.986814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [351.986895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [351.986971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [351.987047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [351.987126] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [351.987266] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [351.987270] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278555, lrc_seqno=278555, guc_id=0, flags=0x73 in no process [-1]
<7> [351.987272] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.987330] ------------[ cut here ]------------
<4> [351.987331] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.987333] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.987406] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.987469] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.987474] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.987477] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.987478] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.987479] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.987484] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.987555] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.987556] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.987558] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.987560] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.987561] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.987562] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.987564] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.987565] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.987567] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.987568] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.987569] PKRU: 55555554
<4> [351.987571] Call Trace:
<4> [351.987572] <TASK>
<4> [351.987576] ? lock_sync+0x100/0x100
<4> [351.987580] ? lock_release+0xd0/0x2b0
<4> [351.987586] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.987591] process_one_work+0x239/0x760
<4> [351.987597] worker_thread+0x200/0x3f0
<4> [351.987600] ? __pfx_worker_thread+0x10/0x10
<4> [351.987602] kthread+0x10d/0x150
<4> [351.987605] ? __pfx_kthread+0x10/0x10
<4> [351.987609] ret_from_fork+0x3d4/0x480
<4> [351.987611] ? __pfx_kthread+0x10/0x10
<4> [351.987614] ret_from_fork_asm+0x1a/0x30
<4> [351.987621] </TASK>
<4> [351.987622] irq event stamp: 1936469
<4> [351.987623] hardirqs last enabled at (1936475): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.987626] hardirqs last disabled at (1936480): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.987628] softirqs last enabled at (1935248): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.987631] softirqs last disabled at (1935239): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.987633] ---[ end trace 0000000000000000 ]---
<7> [351.988703] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [351.988824] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [351.992612] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278557, lrc_seqno=278557, guc_id=0, flags=0x73 in no process [-1]
<7> [351.992614] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [351.992674] ------------[ cut here ]------------
<4> [351.992675] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [351.992677] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [351.992747] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [351.992809] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [351.992813] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [351.992816] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [351.992817] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [351.992818] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [351.992823] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [351.992892] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [351.992893] RSP: 0018:ffffc90007d3fca0 EFLAGS: 00010246
<4> [351.992895] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [351.992897] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [351.992898] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [351.992899] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [351.992900] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [351.992902] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [351.992903] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [351.992904] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [351.992906] PKRU: 55555554
<4> [351.992907] Call Trace:
<4> [351.992908] <TASK>
<4> [351.992912] ? lock_sync+0x100/0x100
<4> [351.992916] ? lock_release+0xd0/0x2b0
<4> [351.992922] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [351.992927] process_one_work+0x239/0x760
<4> [351.992933] worker_thread+0x200/0x3f0
<4> [351.992936] ? __pfx_worker_thread+0x10/0x10
<4> [351.992938] kthread+0x10d/0x150
<4> [351.992941] ? __pfx_kthread+0x10/0x10
<4> [351.992945] ret_from_fork+0x3d4/0x480
<4> [351.992947] ? __pfx_kthread+0x10/0x10
<4> [351.992950] ret_from_fork_asm+0x1a/0x30
<4> [351.992957] </TASK>
<4> [351.992958] irq event stamp: 1937351
<4> [351.992959] hardirqs last enabled at (1937357): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [351.992962] hardirqs last disabled at (1937362): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [351.992964] softirqs last enabled at (1935248): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.992967] softirqs last disabled at (1935239): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [351.992969] ---[ end trace 0000000000000000 ]---
<6> [351.992970] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [351.993040] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [351.993047] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [351.993223] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [351.993436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [351.993528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [351.993622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [351.993708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [351.993793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [351.993877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [351.993961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [351.994049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [351.994136] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [351.994246] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [351.994364] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [351.995390] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.005161] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.005554] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.006630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.006716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.006796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.006872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.006948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.007023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.007099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.007234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.007310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.007384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.007458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.007532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.007607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.007680] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.007752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.007826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.007898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.007969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.008042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.008123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.008215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.008296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.008381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.008462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.008541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.008618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.008695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.008774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.008854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.008930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.009008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.009084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.009174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.009259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.009343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.009428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.009509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.009586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.009663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.009740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.009819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.009896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.009972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.010050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.010135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.010227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.010304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.010379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.010457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.010533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.010608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.010686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.010766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.010846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.010924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.011003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.011080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.011158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.011237] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.011355] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.011359] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278557, lrc_seqno=278557, guc_id=0, flags=0x73 in no process [-1]
<7> [352.011362] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.011419] ------------[ cut here ]------------
<4> [352.011420] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.011422] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:26/3887
<4> [352.011493] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.011557] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.011562] CPU: 4 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.011565] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.011566] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.011568] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.011573] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.011643] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.011644] RSP: 0000:ffffc9000778fca0 EFLAGS: 00010246
<4> [352.011647] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.011648] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.011650] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.011651] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.011652] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.011653] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [352.011655] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.011656] CR2: 00007f796717f000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [352.011658] PKRU: 55555554
<4> [352.011659] Call Trace:
<4> [352.011660] <TASK>
<4> [352.011664] ? lock_sync+0x100/0x100
<4> [352.011669] ? lock_release+0xd0/0x2b0
<4> [352.011674] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.011680] process_one_work+0x239/0x760
<4> [352.011686] worker_thread+0x200/0x3f0
<4> [352.011688] ? __pfx_worker_thread+0x10/0x10
<4> [352.011691] kthread+0x10d/0x150
<4> [352.011694] ? __pfx_kthread+0x10/0x10
<4> [352.011697] ret_from_fork+0x3d4/0x480
<4> [352.011699] ? __pfx_kthread+0x10/0x10
<4> [352.011703] ret_from_fork_asm+0x1a/0x30
<4> [352.011710] </TASK>
<4> [352.011711] irq event stamp: 1462871
<4> [352.011712] hardirqs last enabled at (1462877): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.011715] hardirqs last disabled at (1462882): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.011717] softirqs last enabled at (1461742): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.011720] softirqs last disabled at (1461735): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.011722] ---[ end trace 0000000000000000 ]---
<6> [352.011724] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.011795] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.011801] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.012914] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.013023] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [352.013128] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.013349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.013441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.013534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.013620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.013710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.013798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.013896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.014050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.014209] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.014323] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.014449] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.015473] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.026156] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.026413] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.027398] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.027482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.027564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.027645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.027724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.027803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.027881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.027960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.028040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.028119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.028207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.028286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.028364] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.028442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.028520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.028597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.028674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.028751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.028827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.028913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.028999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.029083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.029173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.029258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.029340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.029422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.029504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.029588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.029672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.029757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.029845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.029932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.030022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.030111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.030208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.030296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.030384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.030468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.030551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.030632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.030715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.030798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.030882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.030963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.031048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.031129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.031218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.031299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.031381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.031461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.031540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.031624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.031704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.031788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.031873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.031957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.032037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.032117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.032208] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.032329] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.032334] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278557, lrc_seqno=278557, guc_id=0, flags=0x73 in no process [-1]
<7> [352.032336] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.032398] ------------[ cut here ]------------
<4> [352.032399] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.032400] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:44/5138
<4> [352.032475] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.032540] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.032545] CPU: 8 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.032548] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.032549] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.032551] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.032557] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.032632] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.032633] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.032636] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.032637] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.032639] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.032640] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.032641] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.032643] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [352.032644] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.032646] CR2: 00007f7965941000 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [352.032647] PKRU: 55555554
<4> [352.032648] Call Trace:
<4> [352.032650] <TASK>
<4> [352.032653] ? lock_sync+0x100/0x100
<4> [352.032658] ? lock_release+0xd0/0x2b0
<4> [352.032664] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.032670] process_one_work+0x239/0x760
<4> [352.032676] worker_thread+0x200/0x3f0
<4> [352.032679] ? __pfx_worker_thread+0x10/0x10
<4> [352.032681] kthread+0x10d/0x150
<4> [352.032684] ? __pfx_kthread+0x10/0x10
<4> [352.032688] ret_from_fork+0x3d4/0x480
<4> [352.032690] ? __pfx_kthread+0x10/0x10
<4> [352.032693] ret_from_fork_asm+0x1a/0x30
<4> [352.032701] </TASK>
<4> [352.032702] irq event stamp: 2067415
<4> [352.032703] hardirqs last enabled at (2067421): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.032706] hardirqs last disabled at (2067426): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.032708] softirqs last enabled at (2066388): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.032711] softirqs last disabled at (2066383): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.032713] ---[ end trace 0000000000000000 ]---
<7> [352.033747] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.033857] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [352.038612] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278558, lrc_seqno=278558, guc_id=0, flags=0x73 in no process [-1]
<7> [352.038617] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.038684] ------------[ cut here ]------------
<4> [352.038685] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.038687] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:44/5138
<4> [352.038762] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.038826] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.038831] CPU: 10 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.038834] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.038835] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.038836] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.038842] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.038913] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.038915] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.038917] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.038919] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.038920] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.038921] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.038922] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.038924] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [352.038925] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.038927] CR2: 00007400ed696450 CR3: 0000000124481005 CR4: 0000000000f72ef0
<4> [352.038928] PKRU: 55555554
<4> [352.038929] Call Trace:
<4> [352.038930] <TASK>
<4> [352.038934] ? lock_sync+0x100/0x100
<4> [352.038939] ? lock_release+0xd0/0x2b0
<4> [352.038945] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.038950] process_one_work+0x239/0x760
<4> [352.038957] worker_thread+0x200/0x3f0
<4> [352.038960] ? __pfx_worker_thread+0x10/0x10
<4> [352.038962] kthread+0x10d/0x150
<4> [352.038965] ? __pfx_kthread+0x10/0x10
<4> [352.038969] ret_from_fork+0x3d4/0x480
<4> [352.038971] ? __pfx_kthread+0x10/0x10
<4> [352.038975] ret_from_fork_asm+0x1a/0x30
<4> [352.038982] </TASK>
<4> [352.038983] irq event stamp: 2068271
<4> [352.038984] hardirqs last enabled at (2068277): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.038988] hardirqs last disabled at (2068282): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.038990] softirqs last enabled at (2066388): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.038992] softirqs last disabled at (2066383): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.038994] ---[ end trace 0000000000000000 ]---
<6> [352.038996] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.039067] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.039074] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.039358] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.039568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.039661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.039755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.039844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.039933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.040021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.040108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.040206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.040289] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.040384] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.040517] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.041528] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.052160] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.052414] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.053440] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.053520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.053598] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.053676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.053753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.053830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.053907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.053985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.054060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.054135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.054223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.054297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.054371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.054444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.054517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.054590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.054663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.054736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.054812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.054895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.054976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.055055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.055135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.055224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.055300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.055377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.055455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.055536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.055616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.055694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.055772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.055851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.055939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.056025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.056111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.056245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.056325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.056403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.056483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.056564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.056644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.056722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.056800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.056876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.056956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.057033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.057110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.057198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.057281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.057360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.057438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.057518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.057595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.057673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.057748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.057825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.057901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.057974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.058116] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.058328] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.058334] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278558, lrc_seqno=278558, guc_id=0, flags=0x73 in no process [-1]
<7> [352.058339] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.058428] ------------[ cut here ]------------
<4> [352.058430] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.058432] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:40/5134
<4> [352.058538] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.058668] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.058677] CPU: 6 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.058681] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.058684] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.058686] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.058695] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.058800] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.058803] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [352.058808] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.058810] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.058813] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.058815] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.058817] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.058820] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [352.058823] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.058826] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [352.058828] PKRU: 55555554
<4> [352.058831] Call Trace:
<4> [352.058833] <TASK>
<4> [352.058841] ? lock_sync+0x100/0x100
<4> [352.058850] ? lock_release+0xd0/0x2b0
<4> [352.058860] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.058870] process_one_work+0x239/0x760
<4> [352.058882] worker_thread+0x200/0x3f0
<4> [352.058887] ? __pfx_worker_thread+0x10/0x10
<4> [352.058891] kthread+0x10d/0x150
<4> [352.058896] ? __pfx_kthread+0x10/0x10
<4> [352.058902] ret_from_fork+0x3d4/0x480
<4> [352.058906] ? __pfx_kthread+0x10/0x10
<4> [352.058911] ret_from_fork_asm+0x1a/0x30
<4> [352.058926] </TASK>
<4> [352.058928] irq event stamp: 2099893
<4> [352.058930] hardirqs last enabled at (2099899): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.058934] hardirqs last disabled at (2099904): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.058938] softirqs last enabled at (2098964): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.058942] softirqs last disabled at (2098957): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.058946] ---[ end trace 0000000000000000 ]---
<6> [352.058949] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.059051] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.059058] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.059470] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.059677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.059768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.059860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.059946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.060031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.060115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.060213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.060300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.060380] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.060474] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.060591] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.061614] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.072160] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.072412] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.073430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.073508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.073584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.073661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.073736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.073811] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.073886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.073960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.074032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.074105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.074187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.074261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.074334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.074407] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.074480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.074554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.074629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.074703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.074777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.074857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.074937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.075015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.075096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.075181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.075260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.075336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.075413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.075491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.075572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.075654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.075735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.075815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.075900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.075983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.076067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.076148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.076242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.076320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.076398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.076476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.076558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.076636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.076712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.076788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.076868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.076948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.077028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.077105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.077194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.077270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.077347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.077427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.077507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.077585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.077661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.077740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.077815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.077891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.077969] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.078083] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.078087] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278558, lrc_seqno=278558, guc_id=0, flags=0x73 in no process [-1]
<7> [352.078090] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.078148] ------------[ cut here ]------------
<4> [352.078153] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.078155] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:40/5134
<4> [352.078227] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.078290] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.078294] CPU: 6 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.078297] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.078298] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.078300] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.078304] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.078374] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.078376] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [352.078378] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.078379] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.078381] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.078382] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.078383] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.078384] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [352.078386] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.078387] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [352.078389] PKRU: 55555554
<4> [352.078390] Call Trace:
<4> [352.078391] <TASK>
<4> [352.078395] ? lock_sync+0x100/0x100
<4> [352.078399] ? lock_release+0xd0/0x2b0
<4> [352.078405] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.078410] process_one_work+0x239/0x760
<4> [352.078416] worker_thread+0x200/0x3f0
<4> [352.078419] ? __pfx_worker_thread+0x10/0x10
<4> [352.078421] kthread+0x10d/0x150
<4> [352.078424] ? __pfx_kthread+0x10/0x10
<4> [352.078428] ret_from_fork+0x3d4/0x480
<4> [352.078430] ? __pfx_kthread+0x10/0x10
<4> [352.078433] ret_from_fork_asm+0x1a/0x30
<4> [352.078440] </TASK>
<4> [352.078441] irq event stamp: 2103011
<4> [352.078443] hardirqs last enabled at (2103017): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.078445] hardirqs last disabled at (2103022): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.078447] softirqs last enabled at (2101800): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.078450] softirqs last disabled at (2101795): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.078452] ---[ end trace 0000000000000000 ]---
<7> [352.079470] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.079576] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [352.083791] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278559, lrc_seqno=278559, guc_id=0, flags=0x73 in no process [-1]
<7> [352.083795] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.083857] ------------[ cut here ]------------
<4> [352.083858] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.083860] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:40/5134
<4> [352.083932] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.083996] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.084000] CPU: 6 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.084003] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.084004] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.084006] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.084011] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.084080] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.084082] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [352.084084] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.084086] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.084087] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.084088] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.084089] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.084091] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [352.084092] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.084094] CR2: 00007f79671c7000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [352.084095] PKRU: 55555554
<4> [352.084096] Call Trace:
<4> [352.084097] <TASK>
<4> [352.084101] ? lock_sync+0x100/0x100
<4> [352.084105] ? lock_release+0xd0/0x2b0
<4> [352.084111] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.084116] process_one_work+0x239/0x760
<4> [352.084122] worker_thread+0x200/0x3f0
<4> [352.084125] ? __pfx_worker_thread+0x10/0x10
<4> [352.084127] kthread+0x10d/0x150
<4> [352.084130] ? __pfx_kthread+0x10/0x10
<4> [352.084134] ret_from_fork+0x3d4/0x480
<4> [352.084136] ? __pfx_kthread+0x10/0x10
<4> [352.084140] ret_from_fork_asm+0x1a/0x30
<4> [352.084147] </TASK>
<4> [352.084148] irq event stamp: 2103887
<4> [352.084156] hardirqs last enabled at (2103903): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.084159] hardirqs last disabled at (2103908): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.084161] softirqs last enabled at (2103898): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.084164] softirqs last disabled at (2103891): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.084166] ---[ end trace 0000000000000000 ]---
<6> [352.084168] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.084238] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.084245] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.084418] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.084627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.084719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.084812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.084901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.084987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.085072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.085164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.085260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.085342] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.085438] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.085554] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.086581] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.097159] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.097408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.098388] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.098474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.098550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.098628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.098704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.098780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.098857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.098932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.099006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.099080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.099161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.099245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.099320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.099397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.099474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.099550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.099625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.099699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.099773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.099853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.099935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.100014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.100094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.100190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.100277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.100355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.100431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.100510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.100591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.100673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.100754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.100833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.100917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.101003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.101088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.101181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.101273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.101351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.101432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.101512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.101594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.101674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.101751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.101829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.101908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.101985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.102061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.102138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.102231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.102318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.102398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.102481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.102560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.102637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.102713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.102791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.102866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.102942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.103020] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.103141] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.103145] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278559, lrc_seqno=278559, guc_id=0, flags=0x73 in no process [-1]
<7> [352.103147] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.103231] ------------[ cut here ]------------
<4> [352.103233] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.103236] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:44/5138
<4> [352.103318] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.103382] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.103387] CPU: 0 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.103390] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.103391] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.103392] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.103398] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.103468] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.103470] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.103472] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.103474] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.103475] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.103476] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.103477] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.103479] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [352.103480] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.103482] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [352.103483] PKRU: 55555554
<4> [352.103484] Call Trace:
<4> [352.103485] <TASK>
<4> [352.103489] ? lock_sync+0x100/0x100
<4> [352.103494] ? lock_release+0xd0/0x2b0
<4> [352.103500] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.103505] process_one_work+0x239/0x760
<4> [352.103511] worker_thread+0x200/0x3f0
<4> [352.103514] ? __pfx_worker_thread+0x10/0x10
<4> [352.103517] kthread+0x10d/0x150
<4> [352.103519] ? __pfx_kthread+0x10/0x10
<4> [352.103523] ret_from_fork+0x3d4/0x480
<4> [352.103525] ? __pfx_kthread+0x10/0x10
<4> [352.103529] ret_from_fork_asm+0x1a/0x30
<4> [352.103536] </TASK>
<4> [352.103537] irq event stamp: 2072485
<4> [352.103538] hardirqs last enabled at (2072491): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.103541] hardirqs last disabled at (2072496): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.103543] softirqs last enabled at (2071468): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.103546] softirqs last disabled at (2071461): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.103548] ---[ end trace 0000000000000000 ]---
<6> [352.103550] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.103621] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.103627] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.104035] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.104246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.104337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.104432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.104520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.104606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.104692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.104779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.104870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.104953] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.105055] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.105176] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.106200] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.116162] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.116411] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.117417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.117502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.117578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.117655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.117733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.117808] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.117884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.117958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.118033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.118106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.118191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.118274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.118347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.118421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.118493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.118567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.118641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.118715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.118789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.118869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.118948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.119027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.119108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.119200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.119286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.119360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.119435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.119513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.119591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.119671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.119753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.119835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.119922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.120006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.120089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.120182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.120274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.120356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.120435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.120512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.120592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.120669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.120745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.120820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.120900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.120975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.121053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.121133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.121229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.121316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.121392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.121470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.121548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.121629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.121707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.121787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.121863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.121939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.122018] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.122132] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.122137] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278559, lrc_seqno=278559, guc_id=0, flags=0x73 in no process [-1]
<7> [352.122139] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.123073] ------------[ cut here ]------------
<4> [352.123074] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.123076] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:44/5138
<4> [352.123164] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.123235] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.123240] CPU: 0 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.123243] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.123244] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.123245] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.123251] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.123320] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.123322] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.123324] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.123326] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.123327] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.123328] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.123330] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.123331] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [352.123333] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.123334] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [352.123336] PKRU: 55555554
<4> [352.123337] Call Trace:
<4> [352.123338] <TASK>
<4> [352.123342] ? lock_sync+0x100/0x100
<4> [352.123346] ? lock_release+0xd0/0x2b0
<4> [352.123352] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.123357] process_one_work+0x239/0x760
<4> [352.123364] worker_thread+0x200/0x3f0
<4> [352.123366] ? __pfx_worker_thread+0x10/0x10
<4> [352.123369] kthread+0x10d/0x150
<4> [352.123372] ? __pfx_kthread+0x10/0x10
<4> [352.123375] ret_from_fork+0x3d4/0x480
<4> [352.123377] ? __pfx_kthread+0x10/0x10
<4> [352.123381] ret_from_fork_asm+0x1a/0x30
<4> [352.123388] </TASK>
<4> [352.123389] irq event stamp: 2075609
<4> [352.123390] hardirqs last enabled at (2075615): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.123393] hardirqs last disabled at (2075620): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.123395] softirqs last enabled at (2074584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.123398] softirqs last disabled at (2074577): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.123400] ---[ end trace 0000000000000000 ]---
<7> [352.123432] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.123550] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [352.124624] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278560, lrc_seqno=278560, guc_id=0, flags=0x73 in no process [-1]
<7> [352.124628] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.124688] ------------[ cut here ]------------
<4> [352.124689] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.124691] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:44/5138
<7> [352.124731] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:672]
<4> [352.124778] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.124842] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.124847] CPU: 0 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.124849] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.124851] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.124852] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.124857] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.124926] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.124928] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.124930] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.124932] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.124933] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.124934] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.124936] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.124937] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [352.124938] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.124940] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [352.124941] PKRU: 55555554
<4> [352.124943] Call Trace:
<4> [352.124944] <TASK>
<4> [352.124947] ? lock_sync+0x100/0x100
<4> [352.124952] ? lock_release+0xd0/0x2b0
<4> [352.124957] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.124962] process_one_work+0x239/0x760
<4> [352.124968] worker_thread+0x200/0x3f0
<4> [352.124971] ? __pfx_worker_thread+0x10/0x10
<4> [352.124974] kthread+0x10d/0x150
<4> [352.124976] ? __pfx_kthread+0x10/0x10
<4> [352.124980] ret_from_fork+0x3d4/0x480
<4> [352.124982] ? __pfx_kthread+0x10/0x10
<4> [352.124986] ret_from_fork_asm+0x1a/0x30
<4> [352.124993] </TASK>
<4> [352.124994] irq event stamp: 2076467
<4> [352.124995] hardirqs last enabled at (2076473): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.124998] hardirqs last disabled at (2076478): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.125000] softirqs last enabled at (2074584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.125003] softirqs last disabled at (2074577): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.125005] ---[ end trace 0000000000000000 ]---
<6> [352.125007] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.125091] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.125098] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.125340] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.125547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.125640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.125791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.125879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.125964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.126048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.126135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.126277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.126364] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.126460] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.126593] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.127672] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.138159] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.138410] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.139403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.139484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.139561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.139638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.139714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.139789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.139862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.139938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.140016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.140091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.140169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.140243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.140317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.140391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.140464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.140538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.140611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.140684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.140760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.140840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.140920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.140999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.141081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.141165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.141248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.141325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.141401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.141479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.141557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.141635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.141714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.141791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.141875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.141959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.142041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.142122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.142211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.142288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.142368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.142446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.142526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.142604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.142680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.142755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.142834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.142910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.142985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.143060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.143138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.143224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.143303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.143385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.143463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.143541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.143618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.143695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.143770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.143850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.143931] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.144048] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.144052] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278560, lrc_seqno=278560, guc_id=0, flags=0x73 in no process [-1]
<7> [352.144054] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.144112] ------------[ cut here ]------------
<4> [352.144114] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.144115] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:40/5134
<4> [352.144207] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.144271] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.144276] CPU: 4 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.144279] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.144280] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.144282] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.144287] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.144356] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.144358] RSP: 0018:ffffc90004f23ca0 EFLAGS: 00010246
<4> [352.144360] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.144361] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.144363] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.144364] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.144365] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.144367] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [352.144368] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.144370] CR2: 00007f796717f000 CR3: 0000000124481001 CR4: 0000000000f72ef0
<4> [352.144371] PKRU: 55555554
<4> [352.144372] Call Trace:
<4> [352.144373] <TASK>
<4> [352.144377] ? lock_sync+0x100/0x100
<4> [352.144382] ? lock_release+0xd0/0x2b0
<4> [352.144387] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.144393] process_one_work+0x239/0x760
<4> [352.144399] worker_thread+0x200/0x3f0
<4> [352.144402] ? __pfx_worker_thread+0x10/0x10
<4> [352.144404] kthread+0x10d/0x150
<4> [352.144407] ? __pfx_kthread+0x10/0x10
<4> [352.144410] ret_from_fork+0x3d4/0x480
<4> [352.144413] ? __pfx_kthread+0x10/0x10
<4> [352.144416] ret_from_fork_asm+0x1a/0x30
<4> [352.144423] </TASK>
<4> [352.144424] irq event stamp: 2107895
<4> [352.144425] hardirqs last enabled at (2107901): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.144428] hardirqs last disabled at (2107906): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.144431] softirqs last enabled at (2107146): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.144433] softirqs last disabled at (2107069): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.144435] ---[ end trace 0000000000000000 ]---
<6> [352.144437] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.144508] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.144514] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.145749] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.145854] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [352.145965] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.146184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.146280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.146375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.146466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.146555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.146642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.146731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.146821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.146903] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.146999] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.147114] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.148142] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.158159] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.158409] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.159433] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.159515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.159590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.159666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.159743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.159822] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.159897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.159973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.160046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.160119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.160206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.160288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.160361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.160435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.160510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.160586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.160660] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.160734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.160807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.160887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.160967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.161046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.161128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.161220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.161309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.161387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.161463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.161542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.161622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.161698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.161775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.161852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.161935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.162019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.162102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.162195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.162287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.162368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.162447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.162523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.162604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.162682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.162760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.162835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.162915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.162996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.163076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.163160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.163250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.163325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.163401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.163480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.163556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.163633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.163708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.163784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.163861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.163938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.164018] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.164135] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.164139] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278560, lrc_seqno=278560, guc_id=0, flags=0x73 in no process [-1]
<7> [352.164142] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.165071] ------------[ cut here ]------------
<4> [352.165072] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.165074] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [352.165161] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.165237] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.165242] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.165245] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.165246] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.165248] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.165254] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.165325] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.165327] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [352.165329] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.165330] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.165332] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.165333] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.165334] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.165336] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [352.165337] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.165339] CR2: 00007f7966f24000 CR3: 000000019fce7003 CR4: 0000000000f72ef0
<4> [352.165340] PKRU: 55555554
<4> [352.165341] Call Trace:
<4> [352.165342] <TASK>
<4> [352.165346] ? lock_sync+0x100/0x100
<4> [352.165351] ? lock_release+0xd0/0x2b0
<4> [352.165356] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.165362] process_one_work+0x239/0x760
<4> [352.165369] worker_thread+0x200/0x3f0
<4> [352.165372] ? __pfx_worker_thread+0x10/0x10
<4> [352.165374] kthread+0x10d/0x150
<4> [352.165377] ? __pfx_kthread+0x10/0x10
<4> [352.165381] ret_from_fork+0x3d4/0x480
<4> [352.165383] ? __pfx_kthread+0x10/0x10
<4> [352.165387] ret_from_fork_asm+0x1a/0x30
<4> [352.165394] </TASK>
<4> [352.165395] irq event stamp: 1941223
<4> [352.165396] hardirqs last enabled at (1941229): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.165399] hardirqs last disabled at (1941234): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.165401] softirqs last enabled at (1940104): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.165404] softirqs last disabled at (1940099): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.165406] ---[ end trace 0000000000000000 ]---
<7> [352.165496] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.165601] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [352.167009] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278569, lrc_seqno=278569, guc_id=0, flags=0x73 in no process [-1]
<7> [352.167012] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.167074] ------------[ cut here ]------------
<4> [352.167075] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.167077] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:38/5132
<4> [352.167166] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.167239] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.167244] CPU: 0 UID: 0 PID: 5132 Comm: kworker/u64:38 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.167247] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.167248] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.167249] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.167255] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.167325] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.167327] RSP: 0000:ffffc90007d3fca0 EFLAGS: 00010246
<4> [352.167329] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.167331] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.167332] RBP: ffffc90007d3fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.167333] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.167334] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.167336] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [352.167337] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.167339] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [352.167341] PKRU: 55555554
<4> [352.167342] Call Trace:
<4> [352.167343] <TASK>
<4> [352.167347] ? lock_sync+0x100/0x100
<4> [352.167351] ? lock_release+0xd0/0x2b0
<4> [352.167357] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.167362] process_one_work+0x239/0x760
<4> [352.167369] worker_thread+0x200/0x3f0
<4> [352.167371] ? __pfx_worker_thread+0x10/0x10
<4> [352.167374] kthread+0x10d/0x150
<4> [352.167377] ? __pfx_kthread+0x10/0x10
<4> [352.167380] ret_from_fork+0x3d4/0x480
<4> [352.167382] ? __pfx_kthread+0x10/0x10
<4> [352.167386] ret_from_fork_asm+0x1a/0x30
<4> [352.167393] </TASK>
<4> [352.167394] irq event stamp: 1942209
<4> [352.167395] hardirqs last enabled at (1942215): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.167398] hardirqs last disabled at (1942220): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.167400] softirqs last enabled at (1940104): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.167403] softirqs last disabled at (1940099): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.167405] ---[ end trace 0000000000000000 ]---
<6> [352.167407] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.167477] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.167484] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.167762] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.167982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.168072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.168196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.168284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.168370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.168458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.168547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.168636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.168720] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.168824] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.168957] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.170033] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.180158] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.180408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.181439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.181518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.181595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.181672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.181749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.181824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.181899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.181974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.182047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.182122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.182207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.182282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.182355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.182428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.182502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.182576] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.182653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.182730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.182805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.182887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.182968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.183047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.183127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.183215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.183293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.183370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.183445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.183524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.183604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.183682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.183760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.183838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.183921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.184005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.184088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.184175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.184257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.184338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.184416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.184492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.184572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.184648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.184724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.184799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.184879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.184962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.185042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.185120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.185208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.185284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.185360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.185438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.185514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.185589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.185663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.185738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.185811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.185885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.185963] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.186078] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.186082] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278569, lrc_seqno=278569, guc_id=0, flags=0x73 in no process [-1]
<7> [352.186085] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.186142] ------------[ cut here ]------------
<4> [352.186143] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.186145] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:26/3887
<4> [352.186232] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.186296] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.186301] CPU: 4 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.186304] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.186305] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.186306] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.186311] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.186381] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.186383] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [352.186385] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.186386] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.186388] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.186389] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.186390] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.186391] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [352.186393] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.186394] CR2: 00007f796717f000 CR3: 0000000124481001 CR4: 0000000000f72ef0
<4> [352.186396] PKRU: 55555554
<4> [352.186397] Call Trace:
<4> [352.186398] <TASK>
<4> [352.186402] ? lock_sync+0x100/0x100
<4> [352.186406] ? lock_release+0xd0/0x2b0
<4> [352.186412] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.186417] process_one_work+0x239/0x760
<4> [352.186423] worker_thread+0x200/0x3f0
<4> [352.186426] ? __pfx_worker_thread+0x10/0x10
<4> [352.186428] kthread+0x10d/0x150
<4> [352.186431] ? __pfx_kthread+0x10/0x10
<4> [352.186435] ret_from_fork+0x3d4/0x480
<4> [352.186436] ? __pfx_kthread+0x10/0x10
<4> [352.186440] ret_from_fork_asm+0x1a/0x30
<4> [352.186447] </TASK>
<4> [352.186448] irq event stamp: 1466787
<4> [352.186449] hardirqs last enabled at (1466793): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.186452] hardirqs last disabled at (1466798): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.186454] softirqs last enabled at (1465864): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.186456] softirqs last disabled at (1465857): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.186459] ---[ end trace 0000000000000000 ]---
<6> [352.186460] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.186534] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.186540] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.187770] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.187877] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [352.187983] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.188194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.188286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.188378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.188466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.188552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.188635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.188721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.188808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.188892] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.188989] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.189106] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.190125] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.200161] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.200413] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.201423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.201514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.201590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.201667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.201743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.201818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.201892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.201965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.202039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.202114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.202204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.202288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.202363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.202438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.202514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.202590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.202665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.202740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.202815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.202897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.202978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.203058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.203136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.203233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.203324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.203405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.203484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.203563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.203643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.203720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.203798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.203876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.203959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.204041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.204124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.204217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.204309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.204389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.204467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.204544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.204623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.204701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.204778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.204853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.204932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.205007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.205083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.205164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.205251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.205325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.205402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.205483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.205561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.205638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.205713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.205791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.205867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.205942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.206021] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.206137] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.206141] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278569, lrc_seqno=278569, guc_id=0, flags=0x73 in no process [-1]
<7> [352.206143] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.206226] ------------[ cut here ]------------
<4> [352.206229] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.206231] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:40/5134
<4> [352.206315] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.206378] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.206383] CPU: 0 UID: 0 PID: 5134 Comm: kworker/u64:40 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.206386] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.206387] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.206388] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.206394] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.206462] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.206464] RSP: 0000:ffffc90004f23ca0 EFLAGS: 00010246
<4> [352.206466] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.206468] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.206469] RBP: ffffc90004f23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.206470] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.206471] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.206473] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [352.206474] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.206476] CR2: 00007f7966f24000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [352.206477] PKRU: 55555554
<4> [352.206478] Call Trace:
<4> [352.206479] <TASK>
<4> [352.206483] ? lock_sync+0x100/0x100
<4> [352.206488] ? lock_release+0xd0/0x2b0
<4> [352.206493] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.206499] process_one_work+0x239/0x760
<4> [352.206505] worker_thread+0x200/0x3f0
<4> [352.206508] ? __pfx_worker_thread+0x10/0x10
<4> [352.206510] kthread+0x10d/0x150
<4> [352.206513] ? __pfx_kthread+0x10/0x10
<4> [352.206516] ret_from_fork+0x3d4/0x480
<4> [352.206519] ? __pfx_kthread+0x10/0x10
<4> [352.206522] ret_from_fork_asm+0x1a/0x30
<4> [352.206529] </TASK>
<4> [352.206530] irq event stamp: 2111385
<4> [352.206531] hardirqs last enabled at (2111391): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.206534] hardirqs last disabled at (2111396): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.206536] softirqs last enabled at (2110274): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.206539] softirqs last disabled at (2110265): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.206541] ---[ end trace 0000000000000000 ]---
<7> [352.207841] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.207947] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [352.308406] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278577, lrc_seqno=278577, guc_id=0, flags=0x73 in no process [-1]
<7> [352.308412] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.308522] ------------[ cut here ]------------
<4> [352.308524] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.308526] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:44/5138
<4> [352.308625] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.308708] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.308712] CPU: 4 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.308715] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.308716] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.308717] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.308723] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.308790] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.308792] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.308794] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.308795] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.308796] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.308797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.308798] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.308800] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [352.308801] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.308802] CR2: 00007f796717f000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [352.308804] PKRU: 55555554
<4> [352.308805] Call Trace:
<4> [352.308806] <TASK>
<4> [352.308810] ? lock_sync+0x100/0x100
<4> [352.308814] ? lock_release+0xd0/0x2b0
<4> [352.308819] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.308824] process_one_work+0x239/0x760
<4> [352.308831] worker_thread+0x200/0x3f0
<4> [352.308833] ? __pfx_worker_thread+0x10/0x10
<4> [352.308835] kthread+0x10d/0x150
<4> [352.308838] ? __pfx_kthread+0x10/0x10
<4> [352.308842] ret_from_fork+0x3d4/0x480
<4> [352.308844] ? __pfx_kthread+0x10/0x10
<4> [352.308847] ret_from_fork_asm+0x1a/0x30
<4> [352.308854] </TASK>
<4> [352.308855] irq event stamp: 2080211
<4> [352.308856] hardirqs last enabled at (2080217): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.308858] hardirqs last disabled at (2080222): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.308860] softirqs last enabled at (2077490): [<ffffffff8133ac33>] kernel_fpu_end+0x53/0x70
<4> [352.308863] softirqs last disabled at (2077488): [<ffffffff8133b344>] kernel_fpu_begin_mask+0xc4/0x120
<4> [352.308865] ---[ end trace 0000000000000000 ]---
<6> [352.308867] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.308937] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.308943] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.309270] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.309498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.309595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.309686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.309773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.309868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.309962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.310053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.310144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.310247] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.310339] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.310470] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.311593] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.322371] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.322640] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.323674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.323759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.323839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.323919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.323998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.324077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.324160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.324240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.324319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.324400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.324481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.324561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.324640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.324717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.324796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.324873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.324950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.325029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.325110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.325205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.325292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.325377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.325462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.325544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.325625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.325707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.325788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.325872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.325961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.326046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.326131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.326223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.326315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.326404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.326494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.326581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.326667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.326757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.326845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.326932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.327021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.327107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.327206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.327295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.327387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.327477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.327568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.327659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.327753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.327846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.327938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.328033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.328130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.328239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.328336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.328434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.328529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.328628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.328736] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.328872] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.328878] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278577, lrc_seqno=278577, guc_id=0, flags=0x73 in no process [-1]
<7> [352.328881] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.328957] ------------[ cut here ]------------
<4> [352.328959] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.328961] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:44/5138
<4> [352.329055] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.329134] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.329140] CPU: 4 UID: 0 PID: 5138 Comm: kworker/u64:44 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.329144] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.329145] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.329147] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.329157] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.329257] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.329260] RSP: 0018:ffffc90004ef3ca0 EFLAGS: 00010246
<4> [352.329263] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.329265] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.329266] RBP: ffffc90004ef3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.329268] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.329270] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.329271] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [352.329274] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.329275] CR2: 00007f796717f000 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [352.329277] PKRU: 55555554
<4> [352.329279] Call Trace:
<4> [352.329280] <TASK>
<4> [352.329286] ? lock_sync+0x100/0x100
<4> [352.329292] ? lock_release+0xd0/0x2b0
<4> [352.329299] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [352.329307] process_one_work+0x239/0x760
<4> [352.329315] worker_thread+0x200/0x3f0
<4> [352.329319] ? __pfx_worker_thread+0x10/0x10
<4> [352.329322] kthread+0x10d/0x150
<4> [352.329326] ? __pfx_kthread+0x10/0x10
<4> [352.329330] ret_from_fork+0x3d4/0x480
<4> [352.329333] ? __pfx_kthread+0x10/0x10
<4> [352.329338] ret_from_fork_asm+0x1a/0x30
<4> [352.329348] </TASK>
<4> [352.329349] irq event stamp: 2083323
<4> [352.329351] hardirqs last enabled at (2083329): [<ffffffff814aacd9>] __up_console_sem+0x79/0xa0
<4> [352.329354] hardirqs last disabled at (2083334): [<ffffffff814aacbe>] __up_console_sem+0x5e/0xa0
<4> [352.329357] softirqs last enabled at (2082358): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.329361] softirqs last disabled at (2082353): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [352.329364] ---[ end trace 0000000000000000 ]---
<6> [352.330533] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [352.330629] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [352.330638] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [352.330654] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [352.330790] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [352.330923] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [352.331171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [352.331288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [352.331403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [352.331512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [352.331616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [352.331718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [352.331818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [352.331922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [352.332021] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [352.332131] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [352.332275] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [352.333299] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [352.343164] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [352.343415] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [352.344482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [352.344561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [352.344636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [352.344710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [352.344784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [352.344858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [352.344931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [352.345004] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [352.345076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [352.345155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [352.345232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [352.345307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [352.345381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [352.345456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [352.345529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [352.345603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [352.345679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [352.345755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [352.345830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [352.345911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [352.345991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [352.346069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [352.346148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [352.346242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [352.346320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [352.346399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [352.346479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [352.346560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [352.346641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [352.346719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [352.346798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [352.346876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [352.346965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [352.347051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [352.347136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [352.347230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [352.347310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [352.347387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [352.347463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [352.347537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [352.347616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [352.347693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [352.347768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [352.347843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [352.347921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [352.347997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [352.348072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [352.348147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [352.348239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [352.348314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [352.348390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [352.348469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [352.348545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [352.348623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [352.348696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [352.348772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [352.348849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [352.348925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [352.349003] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [352.349118] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [352.349122] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=278577, lrc_seqno=278577, guc_id=0, flags=0x73 in no process [-1]
<7> [352.349125] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [352.350040] ------------[ cut here ]------------
<4> [352.350041] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [352.350042] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:26/3887
<4> [352.350117] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp hid_generic cmdlinepart spi_nor asus_nb_wmi mei_pxp mei_hdcp asus_wmi mtd sparse_keymap platform_profile binfmt_misc wmi_bmof kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel rapl intel_cstate r8169 snd_intel_dspcfg snd_hda_codec snd_hda_core video snd_hwdep realtek snd_pcm snd_timer i2c_i801 mei_me idma64 i2c_mux spi_intel_pci snd spi_intel i2c_smbus soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake acpi_pad wmi acpi_tad intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore
<4> [352.350191] nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [352.350196] CPU: 8 UID: 0 PID: 3887 Comm: kworker/u64:26 Tainted: G S U W 7.0.0-rc7-lgci-xe-xe-pw-164582v2-debug+ #1 PREEMPT(lazy)
<4> [352.350199] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [352.350200] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1656 04/18/2024
<4> [352.350202] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [352.350208] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [352.350278] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 06 d6 5e e1 48 89 c6 48 8d 3d 3c 96 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [352.350280] RSP: 0018:ffffc9000778fca0 EFLAGS: 00010246
<4> [352.350282] RAX: ffffffffa1202f0b RBX: 0000000000000000 RCX: 0000000000000000
<4> [352.350284] RDX: ffff888104cf3090 RSI: ffffffffa1202f0b RDI: ffffffffa1003ee0
<4> [352.350285] RBP: ffffc9000778fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [352.350286] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [352.350288] R13: ffff888104cf3090 R14: ffff8881c9bdd018 R15: 00000000ffffffc2
<4> [352.350289] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [352.350291] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [352.350292] CR2: 00007f7967075000 CR3: 000000011fc71001 CR4: 0000000000f72ef0
<4> [352.350294] PKRU: 55555554
<4> [352.350295] Call Trace:
Created at 2026-04-10 05:42:44