Results for igt@xe_pat@pt-caching-update-pat-and-pte

Result: Abort 416 Warning(s)

i915_display_info20 igt_runner20 results20.json results20-xe-load.json guc_logs20.tar i915_display_info_post_exec20 boot20 dmesg20

DetailValue
Duration unknown
Hostname 
shard-bmg-2
Igt-Version 
IGT-Version: 2.4-gc44146a4b (x86_64) (Linux: 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ x86_64)
Out 
Using IGT_SRANDOM=1776799819 for randomisation
Opened device: /dev/dri/card0
Test requirement not met in function __igt_unique____real_main2210, file ../tests/intel/xe_pat.c:2298:
Test requirement: intel_graphics_ver(dev_id) == IP_VER(35, 10)
Last errno: 2, No such file or directory
Starting subtest: pt-caching-update-pat-and-pte
va_bits: 48, num_objs: 16384, spread over pt levels:
[0]: 16384
[1]: 512
[2]: 16
runner: This test was killed due to a kernel taint (0x244).

This test caused an abort condition: Kernel badly tainted (0x244, 0x200) (check dmesg for details):
	TAINT_WARN: WARN_ON has happened.
Err 
Starting subtest: pt-caching-update-pat-and-pte
Received signal SIGQUIT.
Stack trace: 
 #0 [fatal_sig_handler+0x17b]
 #1 [__sigaction+0x50]
 #2 [ioctl+0x3b]
 #3 [drmIoctl+0x30]
 #4 [__syncobj_wait+0x2b]
 #5 [syncobj_wait+0x3f]
 #6 [pt_bind_objects+0xef]
 #7 [pt_caching_test+0x2ea]
 #8 [__igt_unique____real_main2210+0x1443]
 #9 [main+0x3a]
 #10 [__libc_init_first+0x8a]
 #11 [__libc_start_main+0x8b]
 #12 [_start+0x25]
Dmesg

Scroll to first warning
<6> [110.057607] Console: switching to colour dummy device 80x25
<6> [110.057917] [IGT] xe_pat: executing
<6> [110.066321] [IGT] xe_pat: starting subtest pt-caching-update-pat-and-pte
<7> [110.210145] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling AUX_TC2
<7> [110.312979] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling AUX_TC2
<6> [111.242947] xe 0000:03:00.0: [drm] Tile0: GT0:
ASID: 0
Faulted Address: 0x00000002fb426000
FaultType: 0
AccessType: 0
FaultLevel: 2
EngineClass: 3 bcs
EngineInstance: 8
<6> [111.242958] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL
<6> [111.242999] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0
<7> [111.243660] xe 0000:03:00.0: [drm:xe_hw_engine_snapshot_capture [xe]] Tile0: GT0: Proceeding with manual engine snapshot
<6> [111.244154] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249
<5> [111.244168] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23467, lrc_seqno=23467, guc_id=0, flags=0x73 in no process [-1]
<6> [111.424210] xe 0000:03:00.0: [drm] Xe device coredump has been created
<6> [111.424231] xe 0000:03:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data
<4> [111.424234] ------------[ cut here ]------------
<4> [111.424235] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.424237] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:17/6297
<4> [111.424350] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.424443] autofs4 [last unloaded: snd_hda_intel]
<4> [111.424447] CPU: 6 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.424450] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER
<4> [111.424451] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.424453] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.424458] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.424528] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.424530] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.424532] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.424533] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.424535] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.424536] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.424537] R13: ffff888101787710 R14: ffff8881a8888000 R15: 00000000fffffffb
<4> [111.424538] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [111.424539] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.424540] CR2: 0000730469770160 CR3: 000000015b49d003 CR4: 0000000000f72ef0
<4> [111.424542] PKRU: 55555554
<4> [111.424543] Call Trace:
<4> [111.424544] <TASK>
<4> [111.424548] ? lock_sync+0x100/0x100
<4> [111.424553] ? __pfx_autoremove_wake_function+0x10/0x10
<4> [111.424558] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.424564] process_one_work+0x239/0x760
<4> [111.424570] worker_thread+0x200/0x3f0
<4> [111.424573] ? __pfx_worker_thread+0x10/0x10
<4> [111.424575] kthread+0x10d/0x150
<4> [111.424578] ? __pfx_kthread+0x10/0x10
<4> [111.424581] ret_from_fork+0x3d4/0x480
<4> [111.424584] ? __pfx_kthread+0x10/0x10
<4> [111.424587] ret_from_fork_asm+0x1a/0x30
<4> [111.424595] </TASK>
<4> [111.424596] irq event stamp: 788591
<4> [111.424597] hardirqs last enabled at (788597): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.424600] hardirqs last disabled at (788602): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.424602] softirqs last enabled at (787676): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.424604] softirqs last disabled at (787653): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.424606] ---[ end trace 0000000000000000 ]---
<6> [111.424609] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.424677] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.424959] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.425440] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.425905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.426013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.426109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.426198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.426283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.426367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.426450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.426537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.426617] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.426710] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.426837] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.428012] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.438836] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.439113] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.440917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.441003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.441080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.441155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.441230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.441303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.441376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.441448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.441521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.441593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.441667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.441747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.441827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.441908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.441988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.442067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.442148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.442227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.442309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.442398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.442484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.442568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.442653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.442740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.442825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.442908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.442990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.443078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.443167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.443254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.443340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.443423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.443512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.443602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.443691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.443787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.443878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.443966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.444052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.444136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.444223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.444307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.444391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.444473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.444559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.444643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.444726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.444821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.444910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.444995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.445078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.445163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.445245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.445329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.445411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.445499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.445584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.445667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.445757] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.446224] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.446231] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23467, lrc_seqno=23467, guc_id=0, flags=0x73 in no process [-1]
<7> [111.446234] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.446303] ------------[ cut here ]------------
<4> [111.446305] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.446306] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:17/6297
<4> [111.446387] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.446458] autofs4 [last unloaded: snd_hda_intel]
<4> [111.446462] CPU: 6 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.446466] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.446467] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.446469] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.446475] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.446551] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.446553] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.446556] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.446558] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.446559] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.446561] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.446562] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.446564] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [111.446565] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.446567] CR2: 0000730469770160 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [111.446569] PKRU: 55555554
<4> [111.446570] Call Trace:
<4> [111.446572] <TASK>
<4> [111.446576] ? lock_sync+0x100/0x100
<4> [111.446581] ? lock_release+0xd0/0x2b0
<4> [111.446587] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.446593] process_one_work+0x239/0x760
<4> [111.446600] worker_thread+0x200/0x3f0
<4> [111.446603] ? __pfx_worker_thread+0x10/0x10
<4> [111.446606] kthread+0x10d/0x150
<4> [111.446609] ? __pfx_kthread+0x10/0x10
<4> [111.446613] ret_from_fork+0x3d4/0x480
<4> [111.446615] ? __pfx_kthread+0x10/0x10
<4> [111.446619] ret_from_fork_asm+0x1a/0x30
<4> [111.446626] </TASK>
<4> [111.446627] irq event stamp: 791723
<4> [111.446629] hardirqs last enabled at (791729): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.446632] hardirqs last disabled at (791734): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.446634] softirqs last enabled at (790808): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.446637] softirqs last disabled at (790801): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.446640] ---[ end trace 0000000000000000 ]---
<6> [111.446642] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.446716] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.446725] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.448490] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.448612] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.448816] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.449079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.449217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.449357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.449492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.449625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.449763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.449898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.450033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.450163] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.450306] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.450471] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.451875] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.461735] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.462049] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.463372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.463494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.463613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.463749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.463871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.463992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.464113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.464233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.464354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.464475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.464596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.464717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.464843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.464964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.465085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.465206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.465326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.465447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.465568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.465693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.465824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.465949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.466075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.466198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.466322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.466445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.466569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.466696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.466828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.466954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.467080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.467207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.467339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.467471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.467603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.467737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.467865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.467991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.468116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.468240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.468368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.468494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.468619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.468750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.468880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.469005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.469130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.469254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.469379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.469503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.469626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.469758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.469884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.470009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.470134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.470259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.470383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.470507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.470635] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.470985] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.470993] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23467, lrc_seqno=23467, guc_id=0, flags=0x73 in no process [-1]
<7> [111.470997] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.471103] ------------[ cut here ]------------
<4> [111.471104] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.471106] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#15: kworker/u64:157/6437
<4> [111.471220] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.471314] autofs4 [last unloaded: snd_hda_intel]
<4> [111.471320] CPU: 15 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.471324] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.471325] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.471328] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.471336] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.471447] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.471450] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.471453] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.471455] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.471457] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.471459] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.471460] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.471462] FS: 0000000000000000(0000) GS:ffff8888db417000(0000) knlGS:0000000000000000
<4> [111.471465] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.471467] CR2: 0000730469770160 CR3: 000000000344c002 CR4: 0000000000f72ef0
<4> [111.471469] PKRU: 55555554
<4> [111.471470] Call Trace:
<4> [111.471472] <TASK>
<4> [111.471479] ? lock_sync+0x100/0x100
<4> [111.471487] ? lock_release+0xd0/0x2b0
<4> [111.471496] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.471505] process_one_work+0x239/0x760
<4> [111.471515] worker_thread+0x200/0x3f0
<4> [111.471520] ? __pfx_worker_thread+0x10/0x10
<4> [111.471523] kthread+0x10d/0x150
<4> [111.471527] ? __pfx_kthread+0x10/0x10
<4> [111.471532] ret_from_fork+0x3d4/0x480
<4> [111.471535] ? __pfx_kthread+0x10/0x10
<4> [111.471540] ret_from_fork_asm+0x1a/0x30
<4> [111.471552] </TASK>
<4> [111.471554] irq event stamp: 243845
<4> [111.471555] hardirqs last enabled at (243851): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.471559] hardirqs last disabled at (243856): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.471563] softirqs last enabled at (242954): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.471566] softirqs last disabled at (242947): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.471569] ---[ end trace 0000000000000000 ]---
<5> [111.472596] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23522, lrc_seqno=23522, guc_id=0, flags=0x73 in no process [-1]
<7> [111.472601] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.472705] ------------[ cut here ]------------
<4> [111.472707] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.472709] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#15: kworker/u64:157/6437
<4> [111.472828] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.472931] autofs4 [last unloaded: snd_hda_intel]
<4> [111.472937] CPU: 15 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.472941] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.472943] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.472946] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.472953] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.473065] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.473068] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.473071] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.473073] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.473074] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.473076] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.473078] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.473080] FS: 0000000000000000(0000) GS:ffff8888db417000(0000) knlGS:0000000000000000
<4> [111.473082] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.473084] CR2: 0000730469770160 CR3: 000000000344c002 CR4: 0000000000f72ef0
<4> [111.473086] PKRU: 55555554
<4> [111.473088] Call Trace:
<4> [111.473089] <TASK>
<4> [111.473095] ? lock_sync+0x100/0x100
<4> [111.473102] ? lock_release+0xd0/0x2b0
<4> [111.473110] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.473119] process_one_work+0x239/0x760
<4> [111.473129] worker_thread+0x200/0x3f0
<4> [111.473133] ? __pfx_worker_thread+0x10/0x10
<4> [111.473137] kthread+0x10d/0x150
<4> [111.473140] ? __pfx_kthread+0x10/0x10
<4> [111.473146] ret_from_fork+0x3d4/0x480
<4> [111.473148] ? __pfx_kthread+0x10/0x10
<4> [111.473153] ret_from_fork_asm+0x1a/0x30
<4> [111.473165] </TASK>
<4> [111.473167] irq event stamp: 245789
<4> [111.473168] hardirqs last enabled at (245795): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.473172] hardirqs last disabled at (245800): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.473175] softirqs last enabled at (242954): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.473178] softirqs last disabled at (242947): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.473181] ---[ end trace 0000000000000000 ]---
<6> [111.473740] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.473879] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.474059] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.474803] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.474971] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.475132] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.475395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.475533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.475673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.475823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.475958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.476090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.476223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.476359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.476489] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.476632] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.476811] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.478154] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.488732] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.489078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.490093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.490183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.490266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.490348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.490429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.490510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.490591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.490670] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.490765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.490845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.490924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.491003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.491085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.491165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.491244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.491324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.491404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.491484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.491563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.491652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.491742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.491830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.491917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.492001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.492085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.492167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.492250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.492335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.492420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.492504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.492588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.492671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.492770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.492862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.492956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.493045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.493131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.493215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.493297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.493379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.493464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.493548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.493629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.493711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.493807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.493894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.493980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.494066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.494145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.494221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.494298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.494377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.494453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.494531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.494608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.494685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.494769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.494845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.494925] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.495038] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.495046] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23522, lrc_seqno=23522, guc_id=0, flags=0x73 in no process [-1]
<7> [111.495049] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.495108] ------------[ cut here ]------------
<4> [111.495110] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.495112] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:17/6297
<4> [111.495183] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.495254] autofs4 [last unloaded: snd_hda_intel]
<4> [111.495259] CPU: 4 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.495262] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.495263] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.495265] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.495271] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.495343] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.495345] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.495347] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.495349] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.495350] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.495351] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.495352] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.495354] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [111.495355] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.495357] CR2: 00005ee11bb9242c CR3: 00000001b9f46001 CR4: 0000000000f72ef0
<4> [111.495358] PKRU: 55555554
<4> [111.495359] Call Trace:
<4> [111.495361] <TASK>
<4> [111.495365] ? lock_sync+0x100/0x100
<4> [111.495371] ? lock_release+0xd0/0x2b0
<4> [111.495376] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.495382] process_one_work+0x239/0x760
<4> [111.495389] worker_thread+0x200/0x3f0
<4> [111.495392] ? __pfx_worker_thread+0x10/0x10
<4> [111.495394] kthread+0x10d/0x150
<4> [111.495397] ? __pfx_kthread+0x10/0x10
<4> [111.495401] ret_from_fork+0x3d4/0x480
<4> [111.495403] ? __pfx_kthread+0x10/0x10
<4> [111.495406] ret_from_fork_asm+0x1a/0x30
<4> [111.495414] </TASK>
<4> [111.495415] irq event stamp: 795893
<4> [111.495416] hardirqs last enabled at (795899): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.495419] hardirqs last disabled at (795904): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.495422] softirqs last enabled at (794774): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.495425] softirqs last disabled at (794767): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.495427] ---[ end trace 0000000000000000 ]---
<6> [111.495429] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.495502] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.495508] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.496765] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.496880] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.497031] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.497254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.497355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.497455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.497549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.497641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.497739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.497828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.497915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.498002] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.498099] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.498217] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.499244] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.509722] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.509975] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.511000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.511080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.511157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.511234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.511313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.511391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.511469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.511545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.511621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.511697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.511793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.511871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.511948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.512023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.512098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.512173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.512247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.512320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.512394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.512475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.512557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.512639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.512726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.512807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.512885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.512963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.513043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.513127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.513210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.513289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.513370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.513453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.513541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.513628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.513717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.513799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.513880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.513959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.514037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.514113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.514192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.514270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.514347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.514424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.514504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.514581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.514658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.514743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.514822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.514898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.514973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.515055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.515137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.515217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.515294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.515373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.515450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.515526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.515605] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.515721] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.515728] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23522, lrc_seqno=23522, guc_id=0, flags=0x73 in no process [-1]
<7> [111.515730] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.515789] ------------[ cut here ]------------
<4> [111.515790] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.515791] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:155/6435
<4> [111.515862] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.515929] autofs4 [last unloaded: snd_hda_intel]
<4> [111.515933] CPU: 0 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.515936] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.515937] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.515939] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.515944] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.516014] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.516015] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [111.516018] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.516019] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.516020] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.516022] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.516023] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.516024] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.516026] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.516027] CR2: 000073046975303c CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.516029] PKRU: 55555554
<4> [111.516030] Call Trace:
<4> [111.516031] <TASK>
<4> [111.516035] ? lock_sync+0x100/0x100
<4> [111.516040] ? lock_release+0xd0/0x2b0
<4> [111.516046] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.516051] process_one_work+0x239/0x760
<4> [111.516058] worker_thread+0x200/0x3f0
<4> [111.516061] ? __pfx_worker_thread+0x10/0x10
<4> [111.516063] kthread+0x10d/0x150
<4> [111.516066] ? __pfx_kthread+0x10/0x10
<4> [111.516070] ret_from_fork+0x3d4/0x480
<4> [111.516072] ? __pfx_kthread+0x10/0x10
<4> [111.516075] ret_from_fork_asm+0x1a/0x30
<4> [111.516083] </TASK>
<4> [111.516084] irq event stamp: 164545
<4> [111.516085] hardirqs last enabled at (164551): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.516088] hardirqs last disabled at (164556): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.516090] softirqs last enabled at (163442): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.516093] softirqs last disabled at (163435): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.516095] ---[ end trace 0000000000000000 ]---
<5> [111.516451] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23577, lrc_seqno=23577, guc_id=0, flags=0x73 in no process [-1]
<7> [111.516457] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.516649] ------------[ cut here ]------------
<4> [111.516650] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.516653] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:157/6437
<4> [111.516779] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.516883] autofs4 [last unloaded: snd_hda_intel]
<4> [111.516891] CPU: 12 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.516895] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.516898] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.516901] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.516910] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.517037] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.517040] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.517044] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.517046] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.517048] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.517050] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.517052] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.517054] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [111.517056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.517058] CR2: 00007b19c0095018 CR3: 00000001291ae003 CR4: 0000000000f72ef0
<4> [111.517060] PKRU: 55555554
<4> [111.517062] Call Trace:
<4> [111.517063] <TASK>
<4> [111.517070] ? lock_sync+0x100/0x100
<4> [111.517078] ? lock_release+0xd0/0x2b0
<4> [111.517087] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.517096] process_one_work+0x239/0x760
<4> [111.517107] worker_thread+0x200/0x3f0
<4> [111.517111] ? __pfx_worker_thread+0x10/0x10
<4> [111.517114] kthread+0x10d/0x150
<4> [111.517118] ? __pfx_kthread+0x10/0x10
<4> [111.517124] ret_from_fork+0x3d4/0x480
<4> [111.517127] ? __pfx_kthread+0x10/0x10
<4> [111.517132] ret_from_fork_asm+0x1a/0x30
<4> [111.517144] </TASK>
<4> [111.517146] irq event stamp: 248269
<4> [111.517147] hardirqs last enabled at (248275): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.517152] hardirqs last disabled at (248280): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.517155] softirqs last enabled at (246974): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [111.517158] softirqs last disabled at (246972): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [111.517162] ---[ end trace 0000000000000000 ]---
<6> [111.517165] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.517288] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.517299] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.517459] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.517566] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.517674] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.518294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.518381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.518476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.518564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.518650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.518744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.518830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.518917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.518998] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.519094] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.519212] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.520222] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.530718] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.530972] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.531939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.532018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.532096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.532173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.532250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.532330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.532408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.532485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.532561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.532637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.532714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.532787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.532862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.532936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.533010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.533084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.533157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.533231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.533308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.533391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.533473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.533553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.533633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.533716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.533796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.533874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.533952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.534032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.534114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.534193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.534273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.534352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.534437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.534522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.534607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.534693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.534819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.534899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.534980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.535058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.535140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.535219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.535297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.535377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.535462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.535543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.535623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.535702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.535796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.535873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.535950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.536029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.536107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.536183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.536260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.536338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.536416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.536497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.536579] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.536694] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.536698] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23577, lrc_seqno=23577, guc_id=0, flags=0x73 in no process [-1]
<7> [111.536700] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.536783] ------------[ cut here ]------------
<4> [111.536784] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.536785] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.536858] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.536933] autofs4 [last unloaded: snd_hda_intel]
<4> [111.536937] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.536941] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.536942] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.536943] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.536949] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.537019] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.537021] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.537023] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.537025] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.537027] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.537028] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.537029] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.537031] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.537033] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.537034] CR2: 000073046975303c CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.537036] PKRU: 55555554
<4> [111.537038] Call Trace:
<4> [111.537039] <TASK>
<4> [111.537043] ? lock_sync+0x100/0x100
<4> [111.537048] ? lock_release+0xd0/0x2b0
<4> [111.537054] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.537060] process_one_work+0x239/0x760
<4> [111.537066] worker_thread+0x200/0x3f0
<4> [111.537069] ? __pfx_worker_thread+0x10/0x10
<4> [111.537072] kthread+0x10d/0x150
<4> [111.537075] ? __pfx_kthread+0x10/0x10
<4> [111.537078] ret_from_fork+0x3d4/0x480
<4> [111.537081] ? __pfx_kthread+0x10/0x10
<4> [111.537084] ret_from_fork_asm+0x1a/0x30
<4> [111.537092] </TASK>
<4> [111.537093] irq event stamp: 251551
<4> [111.537094] hardirqs last enabled at (251557): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.537097] hardirqs last disabled at (251562): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.537099] softirqs last enabled at (250642): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.537102] softirqs last disabled at (250635): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.537105] ---[ end trace 0000000000000000 ]---
<6> [111.537107] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.537178] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.537184] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.538457] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.538571] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.538678] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.538898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.538993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.539090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.539182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.539269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.539355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.539440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.539526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.539608] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.539704] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.539836] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.540846] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.550714] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.550964] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.551984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.552064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.552143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.552224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.552303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.552381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.552457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.552533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.552609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.552683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.552800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.552875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.552949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.553023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.553097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.553174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.553253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.553332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.553408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.553491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.553572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.553652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.553740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.553824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.553904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.553983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.554061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.554141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.554222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.554301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.554380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.554459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.554548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.554636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.554724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.554807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.554887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.554966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.555047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.555127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.555210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.555290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.555369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.555447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.555527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.555604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.555682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.555767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.555845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.555923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.556003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.556085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.556164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.556242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.556320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.556398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.556475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.556551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.556631] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.556748] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.556752] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23577, lrc_seqno=23577, guc_id=0, flags=0x73 in no process [-1]
<7> [111.556755] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.556815] ------------[ cut here ]------------
<4> [111.556816] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.556817] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [111.556889] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.556955] autofs4 [last unloaded: snd_hda_intel]
<4> [111.556960] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.556963] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.556964] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.556965] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.556971] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.557040] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.557043] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [111.557045] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.557047] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.557048] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.557050] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.557051] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.557053] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [111.557055] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.557056] CR2: 00007b19cb316000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [111.557058] PKRU: 55555554
<4> [111.557059] Call Trace:
<4> [111.557061] <TASK>
<4> [111.557065] ? lock_sync+0x100/0x100
<4> [111.557070] ? lock_release+0xd0/0x2b0
<4> [111.557075] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.557081] process_one_work+0x239/0x760
<4> [111.557088] worker_thread+0x200/0x3f0
<4> [111.557091] ? __pfx_worker_thread+0x10/0x10
<4> [111.557093] kthread+0x10d/0x150
<4> [111.557096] ? __pfx_kthread+0x10/0x10
<4> [111.557100] ret_from_fork+0x3d4/0x480
<4> [111.557102] ? __pfx_kthread+0x10/0x10
<4> [111.557106] ret_from_fork_asm+0x1a/0x30
<4> [111.557113] </TASK>
<4> [111.557114] irq event stamp: 169969
<4> [111.557116] hardirqs last enabled at (169975): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.557119] hardirqs last disabled at (169980): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.557121] softirqs last enabled at (168866): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.557124] softirqs last disabled at (168857): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.557126] ---[ end trace 0000000000000000 ]---
<5> [111.557485] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23632, lrc_seqno=23632, guc_id=0, flags=0x73 in no process [-1]
<7> [111.557488] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.557646] ------------[ cut here ]------------
<4> [111.557648] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.557649] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.557739] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.557813] autofs4 [last unloaded: snd_hda_intel]
<4> [111.557818] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.557822] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.557823] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.557825] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.557832] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.557925] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.557927] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.557929] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.557930] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.557932] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.557933] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.557934] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.557935] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.557937] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.557938] CR2: 000073046975303c CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.557940] PKRU: 55555554
<4> [111.557941] Call Trace:
<4> [111.557942] <TASK>
<4> [111.557947] ? lock_sync+0x100/0x100
<4> [111.557951] ? lock_release+0xd0/0x2b0
<4> [111.557957] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.557963] process_one_work+0x239/0x760
<4> [111.557969] worker_thread+0x200/0x3f0
<4> [111.557972] ? __pfx_worker_thread+0x10/0x10
<4> [111.557974] kthread+0x10d/0x150
<4> [111.557978] ? __pfx_kthread+0x10/0x10
<4> [111.557981] ret_from_fork+0x3d4/0x480
<4> [111.557984] ? __pfx_kthread+0x10/0x10
<4> [111.557988] ret_from_fork_asm+0x1a/0x30
<4> [111.557995] </TASK>
<4> [111.557997] irq event stamp: 252961
<4> [111.557998] hardirqs last enabled at (252967): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.558001] hardirqs last disabled at (252972): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.558003] softirqs last enabled at (251986): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [111.558005] softirqs last disabled at (251984): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [111.558008] ---[ end trace 0000000000000000 ]---
<6> [111.558010] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.558093] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.558104] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.558443] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.558550] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.558657] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.558886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.558977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.559071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.559160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.559249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.559337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.559426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.559514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.559597] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.559694] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.559832] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.560845] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.570711] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.570963] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.571986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.572066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.572143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.572221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.572298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.572375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.572451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.572527] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.572602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.572677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.572763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.572840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.572917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.572995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.573070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.573145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.573219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.573294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.573370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.573454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.573537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.573618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.573703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.573782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.573859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.573936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.574013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.574093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.574174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.574253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.574333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.574413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.574498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.574583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.574669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.574764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.574846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.574930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.575011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.575090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.575171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.575251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.575329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.575407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.575487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.575565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.575642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.575729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.575813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.575892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.575971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.576051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.576130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.576209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.576286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.576364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.576441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.576516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.576597] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.576712] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.576716] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23632, lrc_seqno=23632, guc_id=0, flags=0x73 in no process [-1]
<7> [111.576719] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.576778] ------------[ cut here ]------------
<4> [111.576779] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.576781] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.576852] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.576918] autofs4 [last unloaded: snd_hda_intel]
<4> [111.576922] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.576925] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.576926] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.576927] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.576932] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.577001] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.577002] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.577005] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.577006] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.577007] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.577008] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.577010] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.577011] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.577012] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.577014] CR2: 000073046975303c CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.577015] PKRU: 55555554
<4> [111.577017] Call Trace:
<4> [111.577018] <TASK>
<4> [111.577021] ? lock_sync+0x100/0x100
<4> [111.577026] ? lock_release+0xd0/0x2b0
<4> [111.577032] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.577037] process_one_work+0x239/0x760
<4> [111.577043] worker_thread+0x200/0x3f0
<4> [111.577046] ? __pfx_worker_thread+0x10/0x10
<4> [111.577049] kthread+0x10d/0x150
<4> [111.577051] ? __pfx_kthread+0x10/0x10
<4> [111.577055] ret_from_fork+0x3d4/0x480
<4> [111.577057] ? __pfx_kthread+0x10/0x10
<4> [111.577060] ret_from_fork_asm+0x1a/0x30
<4> [111.577068] </TASK>
<4> [111.577069] irq event stamp: 256145
<4> [111.577070] hardirqs last enabled at (256151): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.577073] hardirqs last disabled at (256156): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.577075] softirqs last enabled at (255230): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.577078] softirqs last disabled at (255223): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.577080] ---[ end trace 0000000000000000 ]---
<6> [111.577081] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.577151] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.577158] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.578405] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.578513] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.578621] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.578844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.578939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.579035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.579125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.579213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.579298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.579384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.579472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.579554] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.579649] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.579775] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.580784] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.590705] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.590954] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.591948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.592027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.592104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.592183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.592262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.592339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.592416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.592492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.592568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.592642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.592723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.592797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.592872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.592947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.593021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.593096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.593171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.593247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.593321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.593404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.593485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.593565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.593646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.593731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.593810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.593890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.593969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.594049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.594130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.594208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.594287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.594366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.594452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.594536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.594619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.594703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.594782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.594859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.594935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.595014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.595098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.595179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.595258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.595334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.595414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.595491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.595567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.595642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.595727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.595801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.595877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.595956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.596036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.596116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.596194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.596273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.596348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.596423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.596506] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.596620] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.596624] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23632, lrc_seqno=23632, guc_id=0, flags=0x73 in no process [-1]
<7> [111.596627] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.596686] ------------[ cut here ]------------
<4> [111.596688] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.596689] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.596776] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.596852] autofs4 [last unloaded: snd_hda_intel]
<4> [111.596857] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.596860] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.596862] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.596863] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.596869] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.596939] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.596941] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.596943] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.596944] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.596946] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.596947] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.596948] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.596950] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.596951] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.596953] CR2: 00007ffdf34afd28 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.596954] PKRU: 55555554
<4> [111.596955] Call Trace:
<4> [111.596957] <TASK>
<4> [111.596960] ? lock_sync+0x100/0x100
<4> [111.596965] ? lock_release+0xd0/0x2b0
<4> [111.596970] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.596976] process_one_work+0x239/0x760
<4> [111.596983] worker_thread+0x200/0x3f0
<4> [111.596985] ? __pfx_worker_thread+0x10/0x10
<4> [111.596988] kthread+0x10d/0x150
<4> [111.596991] ? __pfx_kthread+0x10/0x10
<4> [111.596994] ret_from_fork+0x3d4/0x480
<4> [111.596996] ? __pfx_kthread+0x10/0x10
<4> [111.597000] ret_from_fork_asm+0x1a/0x30
<4> [111.597007] </TASK>
<4> [111.597008] irq event stamp: 800705
<4> [111.597009] hardirqs last enabled at (800711): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.597012] hardirqs last disabled at (800716): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.597015] softirqs last enabled at (799796): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.597017] softirqs last disabled at (799781): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.597019] ---[ end trace 0000000000000000 ]---
<5> [111.597401] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23687, lrc_seqno=23687, guc_id=0, flags=0x73 in no process [-1]
<7> [111.597409] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.597781] ------------[ cut here ]------------
<4> [111.597783] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.597786] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:580/6860
<4> [111.597909] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.598046] autofs4 [last unloaded: snd_hda_intel]
<4> [111.598053] CPU: 12 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.598059] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.598062] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.598064] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.598075] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.598193] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.598196] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.598200] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.598202] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.598204] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.598206] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.598208] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.598210] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [111.598213] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.598215] CR2: 00007b19c0095018 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [111.598217] PKRU: 55555554
<4> [111.598219] Call Trace:
<4> [111.598220] <TASK>
<4> [111.598227] ? lock_sync+0x100/0x100
<4> [111.598236] ? lock_release+0xd0/0x2b0
<4> [111.598245] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.598255] process_one_work+0x239/0x760
<4> [111.598267] worker_thread+0x200/0x3f0
<4> [111.598271] ? __pfx_worker_thread+0x10/0x10
<4> [111.598275] kthread+0x10d/0x150
<4> [111.598279] ? __pfx_kthread+0x10/0x10
<4> [111.598285] ret_from_fork+0x3d4/0x480
<4> [111.598289] ? __pfx_kthread+0x10/0x10
<4> [111.598294] ret_from_fork_asm+0x1a/0x30
<4> [111.598307] </TASK>
<4> [111.598309] irq event stamp: 148449
<4> [111.598311] hardirqs last enabled at (148455): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.598316] hardirqs last disabled at (148460): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.598320] softirqs last enabled at (147688): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.598324] softirqs last disabled at (147681): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.598327] ---[ end trace 0000000000000000 ]---
<6> [111.598331] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.598448] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.598457] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.598513] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.598617] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.599007] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.599215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.599307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.599403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.599493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.599584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.599674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.599772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.599861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.599944] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.600040] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.600158] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.601166] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.611700] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.611950] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.612971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.613051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.613128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.613206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.613282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.613358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.613435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.613511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.613588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.613664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.613772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.613849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.613928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.614006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.614083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.614159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.614234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.614310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.614384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.614467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.614549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.614629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.614713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.614792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.614873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.614955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.615036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.615117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.615200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.615279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.615360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.615439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.615524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.615609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.615697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.615780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.615859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.615936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.616014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.616095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.616180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.616262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.616343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.616421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.616504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.616582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.616660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.616747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.616826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.616902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.616980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.617064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.617147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.617228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.617307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.617389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.617468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.617546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.617627] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.617745] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.617750] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23687, lrc_seqno=23687, guc_id=0, flags=0x73 in no process [-1]
<7> [111.617752] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.617810] ------------[ cut here ]------------
<4> [111.617812] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.617813] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.617884] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.617949] autofs4 [last unloaded: snd_hda_intel]
<4> [111.617953] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.617956] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.617958] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.617959] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.617964] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.618034] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.618036] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.618038] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.618039] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.618041] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.618042] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.618043] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.618044] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.618046] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.618047] CR2: 00007ffdf34afd28 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.618049] PKRU: 55555554
<4> [111.618050] Call Trace:
<4> [111.618051] <TASK>
<4> [111.618055] ? lock_sync+0x100/0x100
<4> [111.618060] ? lock_release+0xd0/0x2b0
<4> [111.618065] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.618071] process_one_work+0x239/0x760
<4> [111.618077] worker_thread+0x200/0x3f0
<4> [111.618079] ? __pfx_worker_thread+0x10/0x10
<4> [111.618082] kthread+0x10d/0x150
<4> [111.618085] ? __pfx_kthread+0x10/0x10
<4> [111.618089] ret_from_fork+0x3d4/0x480
<4> [111.618091] ? __pfx_kthread+0x10/0x10
<4> [111.618094] ret_from_fork_asm+0x1a/0x30
<4> [111.618101] </TASK>
<4> [111.618102] irq event stamp: 805171
<4> [111.618104] hardirqs last enabled at (805177): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.618106] hardirqs last disabled at (805182): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.618109] softirqs last enabled at (804340): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.618111] softirqs last disabled at (804335): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.618113] ---[ end trace 0000000000000000 ]---
<6> [111.618115] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.618186] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.618192] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.619435] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.619541] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.619647] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.619870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.619964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.620060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.620153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.620241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.620328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.620414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.620501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.620583] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.620678] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.620814] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.621824] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.632547] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.632797] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.633790] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.633869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.633946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.634025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.634104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.634184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.634261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.634337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.634413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.634488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.634562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.634636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.634716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.634792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.634867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.634941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.635016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.635089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.635163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.635243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.635324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.635406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.635488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.635567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.635645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.635732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.635809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.635888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.635968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.636046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.636125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.636203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.636287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.636371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.636454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.636534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.636619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.636701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.636780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.636857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.636936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.637014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.637094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.637173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.637256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.637336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.637414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.637492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.637571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.637646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.637730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.637813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.637896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.637976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.638054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.638132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.638208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.638285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.638364] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.638475] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.638479] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23687, lrc_seqno=23687, guc_id=0, flags=0x73 in no process [-1]
<7> [111.638482] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.638540] ------------[ cut here ]------------
<4> [111.638541] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.638543] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:580/6860
<4> [111.638614] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.638680] autofs4 [last unloaded: snd_hda_intel]
<4> [111.638684] CPU: 10 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.638691] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.638692] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.638693] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.638699] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.638768] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.638770] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.638772] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.638774] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.638775] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.638776] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.638777] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.638779] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.638780] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.638782] CR2: 00007b19cb317000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.638783] PKRU: 55555554
<4> [111.638784] Call Trace:
<4> [111.638785] <TASK>
<4> [111.638789] ? lock_sync+0x100/0x100
<4> [111.638794] ? lock_release+0xd0/0x2b0
<4> [111.638799] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.638805] process_one_work+0x239/0x760
<4> [111.638812] worker_thread+0x200/0x3f0
<4> [111.638814] ? __pfx_worker_thread+0x10/0x10
<4> [111.638817] kthread+0x10d/0x150
<4> [111.638820] ? __pfx_kthread+0x10/0x10
<4> [111.638823] ret_from_fork+0x3d4/0x480
<4> [111.638826] ? __pfx_kthread+0x10/0x10
<4> [111.638829] ret_from_fork_asm+0x1a/0x30
<4> [111.638836] </TASK>
<4> [111.638838] irq event stamp: 151967
<4> [111.638839] hardirqs last enabled at (151973): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.638842] hardirqs last disabled at (151978): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.638845] softirqs last enabled at (150882): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.638848] softirqs last disabled at (150873): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.638850] ---[ end trace 0000000000000000 ]---
<7> [111.640347] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.640458] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [111.640557] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23742, lrc_seqno=23742, guc_id=0, flags=0x73 in no process [-1]
<7> [111.640560] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.640619] ------------[ cut here ]------------
<4> [111.640620] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.640621] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:580/6860
<4> [111.640703] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.640778] autofs4 [last unloaded: snd_hda_intel]
<4> [111.640783] CPU: 10 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.640786] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.640787] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.640789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.640795] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.640866] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.640868] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.640870] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.640872] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.640873] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.640875] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.640876] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.640878] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.640880] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.640881] CR2: 00007b19cb317000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.640883] PKRU: 55555554
<4> [111.640884] Call Trace:
<4> [111.640886] <TASK>
<4> [111.640890] ? lock_sync+0x100/0x100
<4> [111.640894] ? lock_release+0xd0/0x2b0
<4> [111.640900] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.640906] process_one_work+0x239/0x760
<4> [111.640912] worker_thread+0x200/0x3f0
<4> [111.640915] ? __pfx_worker_thread+0x10/0x10
<4> [111.640917] kthread+0x10d/0x150
<4> [111.640920] ? __pfx_kthread+0x10/0x10
<4> [111.640924] ret_from_fork+0x3d4/0x480
<4> [111.640927] ? __pfx_kthread+0x10/0x10
<4> [111.640930] ret_from_fork_asm+0x1a/0x30
<4> [111.640938] </TASK>
<4> [111.640939] irq event stamp: 153847
<4> [111.640940] hardirqs last enabled at (153853): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.640943] hardirqs last disabled at (153858): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.640946] softirqs last enabled at (153104): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.640949] softirqs last disabled at (153093): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.640951] ---[ end trace 0000000000000000 ]---
<6> [111.640953] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.641024] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.641031] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.641079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.641405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.641498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.641595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.641707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.641796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.641883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.641969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.642056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.642137] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.642233] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.642350] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.643363] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.653691] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.653943] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.654972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.655051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.655127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.655204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.655281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.655357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.655432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.655510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.655588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.655664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.655780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.655856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.655933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.656010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.656086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.656160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.656236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.656314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.656391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.656475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.656556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.656636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.656730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.656808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.656885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.656962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.657040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.657120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.657201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.657285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.657368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.657449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.657534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.657618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.657705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.657787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.657871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.657952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.658032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.658109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.658189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.658268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.658345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.658421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.658504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.658586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.658666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.658760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.658839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.658915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.658992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.659072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.659149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.659225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.659301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.659383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.659464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.659543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.659623] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.659742] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.659746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23742, lrc_seqno=23742, guc_id=0, flags=0x73 in no process [-1]
<7> [111.659749] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.659807] ------------[ cut here ]------------
<4> [111.659808] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.659810] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.659882] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.659947] autofs4 [last unloaded: snd_hda_intel]
<4> [111.659951] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.659954] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.659955] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.659957] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.659962] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.660033] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.660034] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.660036] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.660038] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.660039] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.660040] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.660041] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.660043] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.660044] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.660046] CR2: 000073046975303c CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.660047] PKRU: 55555554
<4> [111.660048] Call Trace:
<4> [111.660050] <TASK>
<4> [111.660054] ? lock_sync+0x100/0x100
<4> [111.660058] ? lock_release+0xd0/0x2b0
<4> [111.660064] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.660070] process_one_work+0x239/0x760
<4> [111.660076] worker_thread+0x200/0x3f0
<4> [111.660079] ? __pfx_worker_thread+0x10/0x10
<4> [111.660081] kthread+0x10d/0x150
<4> [111.660084] ? __pfx_kthread+0x10/0x10
<4> [111.660088] ret_from_fork+0x3d4/0x480
<4> [111.660090] ? __pfx_kthread+0x10/0x10
<4> [111.660093] ret_from_fork_asm+0x1a/0x30
<4> [111.660100] </TASK>
<4> [111.660102] irq event stamp: 261925
<4> [111.660103] hardirqs last enabled at (261931): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.660105] hardirqs last disabled at (261936): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.660108] softirqs last enabled at (261010): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.660110] softirqs last disabled at (261003): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.660112] ---[ end trace 0000000000000000 ]---
<6> [111.660114] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.660183] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.660189] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.661428] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.661534] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.661640] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.661888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.661981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.662076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.662166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.662253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.662340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.662426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.662515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.662603] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.662706] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.662824] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.663833] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.673689] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.673938] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.674956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.675034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.675110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.675189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.675266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.675346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.675424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.675502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.675578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.675654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.675746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.675821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.675894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.675968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.676043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.676117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.676191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.676266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.676340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.676423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.676507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.676588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.676671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.676765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.676844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.676920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.676996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.677077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.677163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.677247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.677329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.677410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.677496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.677582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.677667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.677765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.677849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.677929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.678008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.678085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.678167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.678246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.678324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.678401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.678481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.678561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.678644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.678733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.678814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.678892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.678972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.679057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.679138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.679218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.679295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.679374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.679451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.679529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.679614] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.679730] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.679734] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23742, lrc_seqno=23742, guc_id=0, flags=0x73 in no process [-1]
<7> [111.679737] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.679796] ------------[ cut here ]------------
<4> [111.679797] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.679799] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:580/6860
<4> [111.679871] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.679936] autofs4 [last unloaded: snd_hda_intel]
<4> [111.679940] CPU: 10 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.679943] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.679944] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.679946] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.679951] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.680020] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.680022] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.680024] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.680026] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.680027] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.680028] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.680029] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.680031] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.680032] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.680034] CR2: 00007b19cb317000 CR3: 00000001291ae006 CR4: 0000000000f72ef0
<4> [111.680035] PKRU: 55555554
<4> [111.680036] Call Trace:
<4> [111.680038] <TASK>
<4> [111.680041] ? lock_sync+0x100/0x100
<4> [111.680046] ? lock_release+0xd0/0x2b0
<4> [111.680051] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.680057] process_one_work+0x239/0x760
<4> [111.680063] worker_thread+0x200/0x3f0
<4> [111.680066] ? __pfx_worker_thread+0x10/0x10
<4> [111.680068] kthread+0x10d/0x150
<4> [111.680071] ? __pfx_kthread+0x10/0x10
<4> [111.680075] ret_from_fork+0x3d4/0x480
<4> [111.680077] ? __pfx_kthread+0x10/0x10
<4> [111.680080] ret_from_fork_asm+0x1a/0x30
<4> [111.680088] </TASK>
<4> [111.680089] irq event stamp: 158361
<4> [111.680090] hardirqs last enabled at (158367): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.680093] hardirqs last disabled at (158372): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.680095] softirqs last enabled at (157264): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.680098] softirqs last disabled at (157255): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.680100] ---[ end trace 0000000000000000 ]---
<5> [111.681319] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23797, lrc_seqno=23797, guc_id=0, flags=0x73 in no process [-1]
<7> [111.681323] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.681387] ------------[ cut here ]------------
<4> [111.681389] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.681390] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:580/6860
<4> [111.681464] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.681528] autofs4 [last unloaded: snd_hda_intel]
<4> [111.681532] CPU: 10 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.681534] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.681536] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.681537] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.681542] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.681612] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.681613] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.681616] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.681617] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.681618] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.681619] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.681620] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.681622] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.681623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.681624] CR2: 00007b19cb317000 CR3: 00000001291ae006 CR4: 0000000000f72ef0
<4> [111.681626] PKRU: 55555554
<4> [111.681627] Call Trace:
<4> [111.681628] <TASK>
<4> [111.681632] ? lock_sync+0x100/0x100
<4> [111.681636] ? lock_release+0xd0/0x2b0
<4> [111.681641] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.681647] process_one_work+0x239/0x760
<4> [111.681653] worker_thread+0x200/0x3f0
<4> [111.681655] ? __pfx_worker_thread+0x10/0x10
<4> [111.681658] kthread+0x10d/0x150
<4> [111.681661] ? __pfx_kthread+0x10/0x10
<4> [111.681664] ret_from_fork+0x3d4/0x480
<4> [111.681667] ? __pfx_kthread+0x10/0x10
<4> [111.681670] ret_from_fork_asm+0x1a/0x30
<4> [111.681677] </TASK>
<4> [111.681682] irq event stamp: 160221
<4> [111.681683] hardirqs last enabled at (160227): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.681686] hardirqs last disabled at (160232): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.681689] softirqs last enabled at (157264): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.681691] softirqs last disabled at (157255): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.681693] ---[ end trace 0000000000000000 ]---
<7> [111.681708] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.681813] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [111.681913] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.681985] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.681991] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.682038] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.682342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.682433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.682525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.682615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.682712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.682799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.682886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.682972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.683054] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.683150] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.683266] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.684275] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.694683] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.694932] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.695954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.696031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.696100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.696172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.696242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.696312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.696381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.696452] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.696525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.696596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.696664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.696749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.696823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.696898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.696971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.697044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.697119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.697196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.697273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.697357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.697440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.697519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.697598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.697682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.697761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.697842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.697921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.698002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.698083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.698162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.698242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.698321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.698405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.698489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.698575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.698655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.698743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.698821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.698901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.698981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.699063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.699143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.699221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.699298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.699379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.699456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.699533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.699610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.699692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.699767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.699845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.699928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.700009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.700088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.700167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.700251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.700330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.700409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.700489] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.700601] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.700606] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23797, lrc_seqno=23797, guc_id=0, flags=0x73 in no process [-1]
<7> [111.700608] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.700667] ------------[ cut here ]------------
<4> [111.700668] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.700670] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:580/6860
<4> [111.700757] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.700822] autofs4 [last unloaded: snd_hda_intel]
<4> [111.700826] CPU: 10 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.700829] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.700830] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.700832] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.700837] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.700907] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.700909] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.700911] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.700912] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.700913] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.700915] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.700916] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.700917] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.700919] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.700920] CR2: 00007b19cb317000 CR3: 00000001291ae006 CR4: 0000000000f72ef0
<4> [111.700921] PKRU: 55555554
<4> [111.700923] Call Trace:
<4> [111.700924] <TASK>
<4> [111.700928] ? lock_sync+0x100/0x100
<4> [111.700932] ? lock_release+0xd0/0x2b0
<4> [111.700938] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.700943] process_one_work+0x239/0x760
<4> [111.700949] worker_thread+0x200/0x3f0
<4> [111.700952] ? __pfx_worker_thread+0x10/0x10
<4> [111.700954] kthread+0x10d/0x150
<4> [111.700957] ? __pfx_kthread+0x10/0x10
<4> [111.700961] ret_from_fork+0x3d4/0x480
<4> [111.700963] ? __pfx_kthread+0x10/0x10
<4> [111.700966] ret_from_fork_asm+0x1a/0x30
<4> [111.700973] </TASK>
<4> [111.700975] irq event stamp: 163319
<4> [111.700976] hardirqs last enabled at (163325): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.700979] hardirqs last disabled at (163330): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.700981] softirqs last enabled at (162410): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.700983] softirqs last disabled at (162399): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.700986] ---[ end trace 0000000000000000 ]---
<6> [111.701921] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.701997] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.702003] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.702018] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.702123] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.702229] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.702435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.702527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.702622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.702719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.702808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.702894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.702981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.703069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.703153] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.703250] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.703369] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.704376] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.714691] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.714940] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.715948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.716028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.716104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.716181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.716258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.716333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.716409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.716485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.716560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.716634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.716719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.716793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.716867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.716942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.717016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.717091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.717168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.717248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.717326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.717410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.717493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.717575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.717656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.717753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.717837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.717919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.717999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.718081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.718162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.718241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.718321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.718401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.718486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.718570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.718655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.718746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.718827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.718905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.718986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.719066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.719148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.719229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.719308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.719385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.719467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.719545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.719623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.719707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.719785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.719865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.719945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.720026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.720106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.720184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.720262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.720340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.720417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.720492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.720571] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.720685] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.720689] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23797, lrc_seqno=23797, guc_id=0, flags=0x73 in no process [-1]
<7> [111.720692] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.720751] ------------[ cut here ]------------
<4> [111.720752] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.720753] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.720826] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.720891] autofs4 [last unloaded: snd_hda_intel]
<4> [111.720895] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.720897] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.720899] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.720900] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.720906] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.720975] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.720976] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.720979] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.720980] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.720981] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.720982] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.720984] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.720985] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.720987] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.720988] CR2: 00007ffdf34afd28 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.720990] PKRU: 55555554
<4> [111.720991] Call Trace:
<4> [111.720992] <TASK>
<4> [111.720996] ? lock_sync+0x100/0x100
<4> [111.721000] ? lock_release+0xd0/0x2b0
<4> [111.721006] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.721011] process_one_work+0x239/0x760
<4> [111.721017] worker_thread+0x200/0x3f0
<4> [111.721020] ? __pfx_worker_thread+0x10/0x10
<4> [111.721023] kthread+0x10d/0x150
<4> [111.721025] ? __pfx_kthread+0x10/0x10
<4> [111.721029] ret_from_fork+0x3d4/0x480
<4> [111.721031] ? __pfx_kthread+0x10/0x10
<4> [111.721035] ret_from_fork_asm+0x1a/0x30
<4> [111.721042] </TASK>
<4> [111.721043] irq event stamp: 810201
<4> [111.721044] hardirqs last enabled at (810207): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.721047] hardirqs last disabled at (810212): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.721049] softirqs last enabled at (809198): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.721052] softirqs last disabled at (809191): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.721054] ---[ end trace 0000000000000000 ]---
<5> [111.722413] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23852, lrc_seqno=23852, guc_id=0, flags=0x73 in no process [-1]
<7> [111.722416] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.722478] ------------[ cut here ]------------
<4> [111.722479] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.722480] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.722549] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.722613] autofs4 [last unloaded: snd_hda_intel]
<4> [111.722616] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.722619] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.722620] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.722621] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.722627] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.722706] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.722708] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.722710] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.722712] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.722713] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.722714] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.722715] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.722717] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.722719] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.722720] CR2: 00007ffdf34afd28 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.722722] PKRU: 55555554
<4> [111.722723] Call Trace:
<4> [111.722724] <TASK>
<4> [111.722728] ? lock_sync+0x100/0x100
<4> [111.722732] ? lock_release+0xd0/0x2b0
<4> [111.722738] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.722744] process_one_work+0x239/0x760
<4> [111.722750] worker_thread+0x200/0x3f0
<4> [111.722753] ? __pfx_worker_thread+0x10/0x10
<4> [111.722756] kthread+0x10d/0x150
<4> [111.722759] ? __pfx_kthread+0x10/0x10
<4> [111.722763] ret_from_fork+0x3d4/0x480
<4> [111.722765] ? __pfx_kthread+0x10/0x10
<4> [111.722768] ret_from_fork_asm+0x1a/0x30
<4> [111.722776] </TASK>
<4> [111.722777] irq event stamp: 812083
<4> [111.722779] hardirqs last enabled at (812089): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.722781] hardirqs last disabled at (812094): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.722784] softirqs last enabled at (811922): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.722787] softirqs last disabled at (811907): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.722789] ---[ end trace 0000000000000000 ]---
<6> [111.722791] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.722861] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.722867] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.723081] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.723185] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.723292] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.723494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.723582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.723683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.723774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.723862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.723948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.724035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.724123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.724206] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.724303] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.724421] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.725431] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.735674] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.735923] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.736867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.736947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.737023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.737100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.737176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.737251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.737327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.737403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.737477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.737551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.737625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.737710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.737784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.737859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.737935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.738008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.738082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.738156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.738231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.738314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.738397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.738479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.738562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.738641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.738728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.738805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.738882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.738962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.739044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.739128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.739211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.739292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.739379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.739465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.739552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.739634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.739724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.739802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.739880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.739959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.740044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.740124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.740203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.740280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.740362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.740439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.740517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.740594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.740676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.740757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.740837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.740917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.740995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.741074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.741151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.741229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.741304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.741380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.741464] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.741577] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.741580] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23852, lrc_seqno=23852, guc_id=0, flags=0x73 in no process [-1]
<7> [111.741583] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.741642] ------------[ cut here ]------------
<4> [111.741644] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.741645] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.741727] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.741804] autofs4 [last unloaded: snd_hda_intel]
<4> [111.741808] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.741811] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.741813] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.741814] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.741819] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.741889] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.741891] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.741893] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.741895] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.741896] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.741897] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.741898] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.741900] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.741901] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.741903] CR2: 00007ffdf34afd28 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.741904] PKRU: 55555554
<4> [111.741905] Call Trace:
<4> [111.741907] <TASK>
<4> [111.741910] ? lock_sync+0x100/0x100
<4> [111.741915] ? lock_release+0xd0/0x2b0
<4> [111.741920] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.741926] process_one_work+0x239/0x760
<4> [111.741932] worker_thread+0x200/0x3f0
<4> [111.741934] ? __pfx_worker_thread+0x10/0x10
<4> [111.741937] kthread+0x10d/0x150
<4> [111.741940] ? __pfx_kthread+0x10/0x10
<4> [111.741943] ret_from_fork+0x3d4/0x480
<4> [111.741945] ? __pfx_kthread+0x10/0x10
<4> [111.741949] ret_from_fork_asm+0x1a/0x30
<4> [111.741956] </TASK>
<4> [111.741957] irq event stamp: 815185
<4> [111.741958] hardirqs last enabled at (815191): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.741961] hardirqs last disabled at (815196): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.741963] softirqs last enabled at (814442): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.741966] softirqs last disabled at (814437): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.741968] ---[ end trace 0000000000000000 ]---
<6> [111.742903] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.742977] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.742983] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.743037] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.743141] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.743247] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.743451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.743540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.743633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.743732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.743820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.743905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.743993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.744080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.744164] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.744258] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.744375] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.745380] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.755670] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.755916] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.756846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.756925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.757003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.757081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.757158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.757236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.757314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.757393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.757471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.757550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.757626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.757712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.757789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.757866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.757942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.758016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.758091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.758165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.758239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.758321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.758402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.758481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.758560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.758638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.758728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.758808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.758886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.758965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.759048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.759131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.759214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.759296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.759382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.759467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.759555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.759640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.759731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.759811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.759889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.759967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.760052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.760134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.760214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.760291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.760371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.760448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.760525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.760601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.760683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.760764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.760842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.760922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.761002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.761081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.761158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.761236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.761313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.761389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.761471] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.761583] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.761587] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23852, lrc_seqno=23852, guc_id=0, flags=0x73 in no process [-1]
<7> [111.761589] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.761648] ------------[ cut here ]------------
<4> [111.761650] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.761651] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.761737] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.761802] autofs4 [last unloaded: snd_hda_intel]
<4> [111.761806] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.761809] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.761810] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.761811] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.761817] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.761887] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.761889] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.761891] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.761892] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.761894] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.761895] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.761896] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.761897] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.761899] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.761900] CR2: 00007ffdf34afd28 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.761902] PKRU: 55555554
<4> [111.761903] Call Trace:
<4> [111.761904] <TASK>
<4> [111.761908] ? lock_sync+0x100/0x100
<4> [111.761912] ? lock_release+0xd0/0x2b0
<4> [111.761918] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.761923] process_one_work+0x239/0x760
<4> [111.761930] worker_thread+0x200/0x3f0
<4> [111.761932] ? __pfx_worker_thread+0x10/0x10
<4> [111.761935] kthread+0x10d/0x150
<4> [111.761938] ? __pfx_kthread+0x10/0x10
<4> [111.761941] ret_from_fork+0x3d4/0x480
<4> [111.761943] ? __pfx_kthread+0x10/0x10
<4> [111.761947] ret_from_fork_asm+0x1a/0x30
<4> [111.761954] </TASK>
<4> [111.761955] irq event stamp: 818291
<4> [111.761956] hardirqs last enabled at (818297): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.761959] hardirqs last disabled at (818302): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.761961] softirqs last enabled at (817548): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.761964] softirqs last disabled at (817543): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.761966] ---[ end trace 0000000000000000 ]---
<5> [111.763323] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23907, lrc_seqno=23907, guc_id=0, flags=0x73 in no process [-1]
<7> [111.763326] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.763390] ------------[ cut here ]------------
<4> [111.763391] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.763392] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.763463] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.763526] autofs4 [last unloaded: snd_hda_intel]
<4> [111.763530] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.763532] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.763533] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.763534] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.763539] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.763607] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.763609] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.763611] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.763613] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.763614] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.763616] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.763617] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.763619] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.763620] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.763622] CR2: 00007ffdf34afd28 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.763623] PKRU: 55555554
<4> [111.763625] Call Trace:
<4> [111.763626] <TASK>
<4> [111.763630] ? lock_sync+0x100/0x100
<4> [111.763634] ? lock_release+0xd0/0x2b0
<4> [111.763640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.763645] process_one_work+0x239/0x760
<4> [111.763652] worker_thread+0x200/0x3f0
<4> [111.763655] ? __pfx_worker_thread+0x10/0x10
<4> [111.763657] kthread+0x10d/0x150
<4> [111.763663] ? __pfx_kthread+0x10/0x10
<4> [111.763667] ret_from_fork+0x3d4/0x480
<4> [111.763670] ? __pfx_kthread+0x10/0x10
<4> [111.763673] ret_from_fork_asm+0x1a/0x30
<4> [111.763681] </TASK>
<4> [111.763682] irq event stamp: 820151
<4> [111.763683] hardirqs last enabled at (820157): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.763686] hardirqs last disabled at (820162): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.763689] softirqs last enabled at (817548): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.763691] softirqs last disabled at (817543): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.763694] ---[ end trace 0000000000000000 ]---
<6> [111.763696] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.763764] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.763770] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.763978] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.764082] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.764187] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.764391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.764480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.764574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.764670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.764761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.764847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.764933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.765020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.765102] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.765196] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.765312] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.766319] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.776666] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.776914] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.777935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.778018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.778096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.778174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.778251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.778326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.778402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.778479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.778557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.778634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.778717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.778793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.778868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.778943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.779017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.779090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.779164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.779238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.779312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.779393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.779476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.779555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.779636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.779723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.779807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.779887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.779966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.780046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.780126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.780205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.780284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.780364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.780452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.780541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.780627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.780716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.780797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.780876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.780955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.781036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.781120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.781200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.781279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.781356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.781438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.781516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.781595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.781680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.781761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.781839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.781917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.781998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.782080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.782161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.782239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.782318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.782395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.782472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.782552] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.783509] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.783514] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23907, lrc_seqno=23907, guc_id=0, flags=0x73 in no process [-1]
<7> [111.783516] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.783581] ------------[ cut here ]------------
<4> [111.783582] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.783584] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:17/6297
<4> [111.783661] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.783729] autofs4 [last unloaded: snd_hda_intel]
<4> [111.783733] CPU: 8 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.783736] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.783738] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.783739] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.783745] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.783815] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.783817] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.783820] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.783821] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.783823] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.783824] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.783826] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.783827] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.783829] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.783830] CR2: 00007ffdf34afd28 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.783832] PKRU: 55555554
<4> [111.783833] Call Trace:
<4> [111.783835] <TASK>
<4> [111.783839] ? lock_sync+0x100/0x100
<4> [111.783844] ? lock_release+0xd0/0x2b0
<4> [111.783849] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.783855] process_one_work+0x239/0x760
<4> [111.783861] worker_thread+0x200/0x3f0
<4> [111.783864] ? __pfx_worker_thread+0x10/0x10
<4> [111.783867] kthread+0x10d/0x150
<4> [111.783870] ? __pfx_kthread+0x10/0x10
<4> [111.783874] ret_from_fork+0x3d4/0x480
<4> [111.783876] ? __pfx_kthread+0x10/0x10
<4> [111.783880] ret_from_fork_asm+0x1a/0x30
<4> [111.783887] </TASK>
<4> [111.783889] irq event stamp: 823253
<4> [111.783890] hardirqs last enabled at (823259): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.783893] hardirqs last disabled at (823264): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.783895] softirqs last enabled at (822512): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.783898] softirqs last disabled at (822505): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.783900] ---[ end trace 0000000000000000 ]---
<6> [111.783902] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.783973] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.783979] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.783993] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.784097] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.784387] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.784597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.784700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.784799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.784891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.784980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.785066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.785152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.785239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.785321] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.785417] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.785535] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.786545] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.796668] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.796917] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.797946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.798026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.798105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.798183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.798260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.798336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.798411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.798486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.798561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.798635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.798722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.798797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.798871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.798945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.799018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.799092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.799166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.799240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.799314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.799397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.799477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.799557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.799642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.799735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.799815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.799892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.799969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.800048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.800129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.800207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.800285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.800363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.800449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.800533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.800616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.800715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.800800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.800882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.800964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.801044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.801126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.801206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.801284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.801361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.801445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.801527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.801606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.801697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.801775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.801851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.801927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.802006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.802086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.802167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.802246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.802325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.802403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.802479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.802559] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.802677] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.802681] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23907, lrc_seqno=23907, guc_id=0, flags=0x73 in no process [-1]
<7> [111.802684] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.802742] ------------[ cut here ]------------
<4> [111.802743] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.802745] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.802817] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.802883] autofs4 [last unloaded: snd_hda_intel]
<4> [111.802887] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.802890] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.802891] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.802892] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.802897] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.802968] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.802969] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.802972] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.802973] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.802974] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.802975] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.802977] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.802978] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.802980] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.802981] CR2: 00007b19cb319000 CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.802983] PKRU: 55555554
<4> [111.802984] Call Trace:
<4> [111.802985] <TASK>
<4> [111.802989] ? lock_sync+0x100/0x100
<4> [111.802993] ? lock_release+0xd0/0x2b0
<4> [111.802999] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.803004] process_one_work+0x239/0x760
<4> [111.803010] worker_thread+0x200/0x3f0
<4> [111.803013] ? __pfx_worker_thread+0x10/0x10
<4> [111.803016] kthread+0x10d/0x150
<4> [111.803018] ? __pfx_kthread+0x10/0x10
<4> [111.803022] ret_from_fork+0x3d4/0x480
<4> [111.803024] ? __pfx_kthread+0x10/0x10
<4> [111.803027] ret_from_fork_asm+0x1a/0x30
<4> [111.803035] </TASK>
<4> [111.803036] irq event stamp: 269643
<4> [111.803037] hardirqs last enabled at (269649): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.803040] hardirqs last disabled at (269654): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.803042] softirqs last enabled at (268728): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.803045] softirqs last disabled at (268721): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.803047] ---[ end trace 0000000000000000 ]---
<5> [111.804234] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23962, lrc_seqno=23962, guc_id=0, flags=0x73 in no process [-1]
<7> [111.804237] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.804302] ------------[ cut here ]------------
<4> [111.804303] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.804304] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.804375] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.804438] autofs4 [last unloaded: snd_hda_intel]
<4> [111.804442] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.804444] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.804445] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.804447] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.804452] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.804520] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.804522] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.804524] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.804525] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.804526] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.804527] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.804529] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.804530] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.804531] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.804533] CR2: 00007b19cb319000 CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [111.804534] PKRU: 55555554
<4> [111.804535] Call Trace:
<4> [111.804536] <TASK>
<4> [111.804540] ? lock_sync+0x100/0x100
<4> [111.804544] ? lock_release+0xd0/0x2b0
<4> [111.804549] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.804555] process_one_work+0x239/0x760
<4> [111.804561] worker_thread+0x200/0x3f0
<4> [111.804563] ? __pfx_worker_thread+0x10/0x10
<4> [111.804566] kthread+0x10d/0x150
<4> [111.804568] ? __pfx_kthread+0x10/0x10
<4> [111.804572] ret_from_fork+0x3d4/0x480
<4> [111.804574] ? __pfx_kthread+0x10/0x10
<4> [111.804577] ret_from_fork_asm+0x1a/0x30
<4> [111.804585] </TASK>
<4> [111.804586] irq event stamp: 271513
<4> [111.804587] hardirqs last enabled at (271519): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.804589] hardirqs last disabled at (271524): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.804591] softirqs last enabled at (268728): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.804594] softirqs last disabled at (268721): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.804596] ---[ end trace 0000000000000000 ]---
<6> [111.804598] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.804670] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.804678] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.804830] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.804935] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.805041] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.805247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.805340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.805436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.805527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.805616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.805715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.805806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.805899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.805984] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.806082] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.806200] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.807207] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.817657] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.817906] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.818940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.819019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.819095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.819173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.819251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.819327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.819404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.819480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.819557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.819636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.819750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.819826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.819901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.819976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.820051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.820125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.820199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.820272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.820347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.820430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.820512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.820592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.820679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.820758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.820840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.820922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.821001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.821083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.821164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.821243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.821324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.821403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.821491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.821576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.821664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.821745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.821830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.821914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.821995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.822075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.822156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.822236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.822314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.822392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.822473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.822551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.822631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.822724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.822806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.822885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.822963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.823044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.823123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.823201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.823278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.823356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.823433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.823510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.823590] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.823959] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.823963] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23962, lrc_seqno=23962, guc_id=0, flags=0x73 in no process [-1]
<7> [111.823966] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.824026] ------------[ cut here ]------------
<4> [111.824027] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.824028] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [111.824099] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.824165] autofs4 [last unloaded: snd_hda_intel]
<4> [111.824169] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.824172] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.824174] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.824175] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.824181] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.824252] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.824254] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.824256] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.824258] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.824259] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.824261] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.824262] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.824264] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.824266] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.824267] CR2: 00007b19cb317000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.824269] PKRU: 55555554
<4> [111.824270] Call Trace:
<4> [111.824271] <TASK>
<4> [111.824276] ? lock_sync+0x100/0x100
<4> [111.824280] ? lock_release+0xd0/0x2b0
<4> [111.824286] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.824292] process_one_work+0x239/0x760
<4> [111.824298] worker_thread+0x200/0x3f0
<4> [111.824302] ? __pfx_worker_thread+0x10/0x10
<4> [111.824304] kthread+0x10d/0x150
<4> [111.824307] ? __pfx_kthread+0x10/0x10
<4> [111.824311] ret_from_fork+0x3d4/0x480
<4> [111.824313] ? __pfx_kthread+0x10/0x10
<4> [111.824317] ret_from_fork_asm+0x1a/0x30
<4> [111.824325] </TASK>
<4> [111.824326] irq event stamp: 827139
<4> [111.824327] hardirqs last enabled at (827145): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.824330] hardirqs last disabled at (827150): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.824333] softirqs last enabled at (826224): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.824335] softirqs last disabled at (826215): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.824338] ---[ end trace 0000000000000000 ]---
<6> [111.824340] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.824411] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.824417] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.825712] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.825817] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.825923] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.826126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.826214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.826309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.826400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.826489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.826577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.826667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.826755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.826836] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.826930] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.827047] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.828055] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.838654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.838903] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.840027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.840107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.840184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.840261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.840336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.840412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.840486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.840560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.840635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.840727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.840805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.840881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.840956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.841031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.841106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.841180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.841255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.841330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.841404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.841487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.841569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.841654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.841738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.841817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.841896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.841974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.842051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.842132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.842212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.842291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.842371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.842454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.842543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.842630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.842727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.842809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.842890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.842969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.843046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.843122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.843202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.843283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.843363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.843442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.843524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.843602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.843687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.843765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.843843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.843920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.843996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.844075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.844151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.844228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.844307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.844387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.844465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.844541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.844620] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.844742] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.844746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=23962, lrc_seqno=23962, guc_id=0, flags=0x73 in no process [-1]
<7> [111.844749] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.844807] ------------[ cut here ]------------
<4> [111.844808] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.844810] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [111.844881] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.844947] autofs4 [last unloaded: snd_hda_intel]
<4> [111.844950] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.844953] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.844954] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.844956] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.844961] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.845031] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.845033] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.845035] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.845037] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.845038] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.845039] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.845040] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.845042] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.845043] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.845044] CR2: 00007b19cb317000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.845046] PKRU: 55555554
<4> [111.845047] Call Trace:
<4> [111.845048] <TASK>
<4> [111.845052] ? lock_sync+0x100/0x100
<4> [111.845056] ? lock_release+0xd0/0x2b0
<4> [111.845062] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.845068] process_one_work+0x239/0x760
<4> [111.845074] worker_thread+0x200/0x3f0
<4> [111.845076] ? __pfx_worker_thread+0x10/0x10
<4> [111.845079] kthread+0x10d/0x150
<4> [111.845082] ? __pfx_kthread+0x10/0x10
<4> [111.845085] ret_from_fork+0x3d4/0x480
<4> [111.845087] ? __pfx_kthread+0x10/0x10
<4> [111.845091] ret_from_fork_asm+0x1a/0x30
<4> [111.845098] </TASK>
<4> [111.845099] irq event stamp: 830239
<4> [111.845100] hardirqs last enabled at (830245): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.845103] hardirqs last disabled at (830250): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.845105] softirqs last enabled at (829130): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.845108] softirqs last disabled at (829121): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.845110] ---[ end trace 0000000000000000 ]---
<5> [111.846349] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24017, lrc_seqno=24017, guc_id=0, flags=0x73 in no process [-1]
<7> [111.846352] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.846430] ------------[ cut here ]------------
<4> [111.846431] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.846433] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [111.846504] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.846568] autofs4 [last unloaded: snd_hda_intel]
<4> [111.846572] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.846574] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.846575] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.846577] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.846582] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.846655] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.846657] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.846659] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.846660] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.846662] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.846663] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.846664] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.846666] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.846667] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.846669] CR2: 00007b19cb317000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.846670] PKRU: 55555554
<4> [111.846671] Call Trace:
<4> [111.846673] <TASK>
<4> [111.846677] ? lock_sync+0x100/0x100
<4> [111.846681] ? lock_release+0xd0/0x2b0
<4> [111.846687] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.846692] process_one_work+0x239/0x760
<4> [111.846699] worker_thread+0x200/0x3f0
<4> [111.846702] ? __pfx_worker_thread+0x10/0x10
<4> [111.846704] kthread+0x10d/0x150
<4> [111.846707] ? __pfx_kthread+0x10/0x10
<4> [111.846711] ret_from_fork+0x3d4/0x480
<4> [111.846713] ? __pfx_kthread+0x10/0x10
<4> [111.846717] ret_from_fork_asm+0x1a/0x30
<4> [111.846724] </TASK>
<4> [111.846726] irq event stamp: 832525
<4> [111.846727] hardirqs last enabled at (832531): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.846730] hardirqs last disabled at (832536): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.846732] softirqs last enabled at (829130): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.846735] softirqs last disabled at (829121): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.846738] ---[ end trace 0000000000000000 ]---
<7> [111.846752] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.846856] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [111.846958] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.847029] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.847037] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.847091] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.847352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.847490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.847630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.847777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.847912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.848045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.848178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.848313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.848445] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.848588] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.848790] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.850205] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.860672] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.861024] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.862201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.862323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.862443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.862564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.862694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.862816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.862938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.863059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.863180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.863301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.863422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.863543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.863686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.863774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.863855] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.863933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.864012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.864088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.864164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.864249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.864332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.864411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.864491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.864567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.864648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.864724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.864801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.864880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.864959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.865037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.865119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.865202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.865289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.865375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.865460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.865541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.865622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.865715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.865796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.865874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.865955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.866034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.866113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.866189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.866270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.866348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.866425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.866502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.866582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.866671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.866752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.866833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.866911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.866989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.867066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.867144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.867222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.867301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.867382] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.867494] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.867499] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24017, lrc_seqno=24017, guc_id=0, flags=0x73 in no process [-1]
<7> [111.867501] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.867559] ------------[ cut here ]------------
<4> [111.867560] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.867562] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [111.867634] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.867703] autofs4 [last unloaded: snd_hda_intel]
<4> [111.867708] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.867711] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.867712] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.867713] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.867718] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.867788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.867790] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.867792] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.867794] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.867795] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.867796] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.867797] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.867799] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.867800] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.867802] CR2: 00007b19cb31a000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.867803] PKRU: 55555554
<4> [111.867804] Call Trace:
<4> [111.867806] <TASK>
<4> [111.867809] ? lock_sync+0x100/0x100
<4> [111.867814] ? lock_release+0xd0/0x2b0
<4> [111.867820] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.867826] process_one_work+0x239/0x760
<4> [111.867832] worker_thread+0x200/0x3f0
<4> [111.867836] ? __pfx_worker_thread+0x10/0x10
<4> [111.867838] kthread+0x10d/0x150
<4> [111.867841] ? __pfx_kthread+0x10/0x10
<4> [111.867845] ret_from_fork+0x3d4/0x480
<4> [111.867847] ? __pfx_kthread+0x10/0x10
<4> [111.867851] ret_from_fork_asm+0x1a/0x30
<4> [111.867858] </TASK>
<4> [111.867860] irq event stamp: 171261
<4> [111.867861] hardirqs last enabled at (171267): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.867864] hardirqs last disabled at (171272): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.867867] softirqs last enabled at (170354): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.867869] softirqs last disabled at (170349): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.867871] ---[ end trace 0000000000000000 ]---
<6> [111.867873] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.867943] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.867949] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.869187] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.869307] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.869424] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.869628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.869730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.869823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.869909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.869995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.870080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.870166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.870254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.870337] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.870433] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.870556] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.871912] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.882648] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.882957] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.884133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.884254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.884373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.884494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.884615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.884744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.884866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.884986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.885107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.885228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.885348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.885479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.885567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.885678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.885756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.885834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.885910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.885986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.886063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.886148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.886231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.886311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.886390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.886468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.886546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.886627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.886721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.886802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.886883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.886966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.887050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.887132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.887218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.887304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.887389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.887471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.887552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.887631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.887721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.887798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.887882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.887965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.888047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.888126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.888209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.888287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.888366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.888443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.888523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.888599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.888683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.888763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.888845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.888928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.889009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.889090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.889167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.889245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.889326] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.889439] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.889444] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24017, lrc_seqno=24017, guc_id=0, flags=0x73 in no process [-1]
<7> [111.889446] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.889505] ------------[ cut here ]------------
<4> [111.889507] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.889508] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:17/6297
<4> [111.889580] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.889649] autofs4 [last unloaded: snd_hda_intel]
<4> [111.889654] CPU: 6 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.889657] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.889658] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.889659] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.889665] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.889736] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.889738] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.889740] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.889742] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.889743] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.889744] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.889745] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.889747] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [111.889748] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.889750] CR2: 00007b19cb316000 CR3: 000000015b49d003 CR4: 0000000000f72ef0
<4> [111.889751] PKRU: 55555554
<4> [111.889752] Call Trace:
<4> [111.889754] <TASK>
<4> [111.889757] ? lock_sync+0x100/0x100
<4> [111.889762] ? lock_release+0xd0/0x2b0
<4> [111.889768] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.889773] process_one_work+0x239/0x760
<4> [111.889780] worker_thread+0x200/0x3f0
<4> [111.889783] ? __pfx_worker_thread+0x10/0x10
<4> [111.889785] kthread+0x10d/0x150
<4> [111.889788] ? __pfx_kthread+0x10/0x10
<4> [111.889792] ret_from_fork+0x3d4/0x480
<4> [111.889794] ? __pfx_kthread+0x10/0x10
<4> [111.889797] ret_from_fork_asm+0x1a/0x30
<4> [111.889805] </TASK>
<4> [111.889806] irq event stamp: 837013
<4> [111.889807] hardirqs last enabled at (837019): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.889810] hardirqs last disabled at (837024): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.889813] softirqs last enabled at (835934): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.889815] softirqs last disabled at (835927): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.889817] ---[ end trace 0000000000000000 ]---
<5> [111.890452] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24072, lrc_seqno=24072, guc_id=0, flags=0x73 in no process [-1]
<7> [111.890456] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.890639] ------------[ cut here ]------------
<4> [111.890642] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.890644] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [111.890722] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.890796] autofs4 [last unloaded: snd_hda_intel]
<4> [111.890802] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.890805] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.890806] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.890808] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.890814] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.890894] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.890896] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.890898] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.890900] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.890901] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.890902] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.890903] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.890905] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.890906] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.890908] CR2: 00007b19cb31a000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.890909] PKRU: 55555554
<4> [111.890910] Call Trace:
<4> [111.890912] <TASK>
<4> [111.890915] ? lock_sync+0x100/0x100
<4> [111.890920] ? lock_release+0xd0/0x2b0
<4> [111.890926] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.890932] process_one_work+0x239/0x760
<4> [111.890939] worker_thread+0x200/0x3f0
<4> [111.890941] ? __pfx_worker_thread+0x10/0x10
<4> [111.890944] kthread+0x10d/0x150
<4> [111.890947] ? __pfx_kthread+0x10/0x10
<4> [111.890950] ret_from_fork+0x3d4/0x480
<4> [111.890952] ? __pfx_kthread+0x10/0x10
<4> [111.890956] ret_from_fork_asm+0x1a/0x30
<4> [111.890964] </TASK>
<4> [111.890965] irq event stamp: 172857
<4> [111.890967] hardirqs last enabled at (172863): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.890970] hardirqs last disabled at (172868): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.890972] softirqs last enabled at (171792): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [111.890974] softirqs last disabled at (171790): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [111.890977] ---[ end trace 0000000000000000 ]---
<6> [111.890979] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.891062] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.891073] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.891580] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.891705] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.891812] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.892015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.892105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.892199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.892290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.892379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.892466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.892553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.892678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.892763] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.892862] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.892981] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.893992] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.904673] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.904924] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.905942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.906024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.906102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.906178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.906254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.906329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.906404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.906479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.906553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.906628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.906715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.906789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.906863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.906937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.907011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.907084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.907158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.907232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.907305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.907387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.907469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.907549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.907636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.907718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.907797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.907875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.907952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.908035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.908119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.908201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.908282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.908362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.908448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.908533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.908618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.908713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.908794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.908873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.908952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.909028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.909108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.909187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.909270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.909351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.909435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.909515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.909593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.909680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.909759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.909836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.909913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.909992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.910071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.910153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.910235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.910315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.910393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.910470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.910551] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.910669] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.910673] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24072, lrc_seqno=24072, guc_id=0, flags=0x73 in no process [-1]
<7> [111.910676] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.910734] ------------[ cut here ]------------
<4> [111.910735] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.910737] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [111.910810] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.910875] autofs4 [last unloaded: snd_hda_intel]
<4> [111.910878] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.910881] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.910882] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.910884] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.910889] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.910959] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.910961] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.910963] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.910964] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.910966] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.910967] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.910968] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.910969] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.910971] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.910972] CR2: 00007b19cb31a000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.910974] PKRU: 55555554
<4> [111.910975] Call Trace:
<4> [111.910976] <TASK>
<4> [111.910980] ? lock_sync+0x100/0x100
<4> [111.910984] ? lock_release+0xd0/0x2b0
<4> [111.910990] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.910995] process_one_work+0x239/0x760
<4> [111.911001] worker_thread+0x200/0x3f0
<4> [111.911004] ? __pfx_worker_thread+0x10/0x10
<4> [111.911006] kthread+0x10d/0x150
<4> [111.911009] ? __pfx_kthread+0x10/0x10
<4> [111.911013] ret_from_fork+0x3d4/0x480
<4> [111.911015] ? __pfx_kthread+0x10/0x10
<4> [111.911018] ret_from_fork_asm+0x1a/0x30
<4> [111.911026] </TASK>
<4> [111.911027] irq event stamp: 176077
<4> [111.911028] hardirqs last enabled at (176083): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.911031] hardirqs last disabled at (176088): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.911033] softirqs last enabled at (175268): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.911036] softirqs last disabled at (175263): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.911038] ---[ end trace 0000000000000000 ]---
<6> [111.911040] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.911109] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.911114] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.911285] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.911486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.911575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.911675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.911763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.911855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.911943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.912031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.912118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.912201] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.912298] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.912417] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.913424] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.923635] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.923885] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.924946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.925025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.925102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.925179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.925258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.925337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.925415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.925490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.925567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.925645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.925721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.925798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.925872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.925951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.926029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.926106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.926182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.926257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.926332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.926414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.926495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.926575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.926666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.926746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.926825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.926903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.926980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.927061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.927142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.927220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.927299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.927378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.927462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.927546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.927634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.927718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.927804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.927885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.927965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.928042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.928123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.928201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.928282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.928363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.928447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.928526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.928604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.928689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.928768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.928844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.928920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.928999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.929079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.929161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.929240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.929319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.929396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.929474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.929554] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.929668] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.929672] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24072, lrc_seqno=24072, guc_id=0, flags=0x73 in no process [-1]
<7> [111.929674] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.929732] ------------[ cut here ]------------
<4> [111.929733] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.929735] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [111.929806] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.929870] autofs4 [last unloaded: snd_hda_intel]
<4> [111.929873] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.929876] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.929877] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.929879] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.929883] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.929953] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.929955] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.929957] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.929958] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.929959] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.929961] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.929962] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.929963] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.929965] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.929966] CR2: 00007b19cb31a000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.929968] PKRU: 55555554
<4> [111.929969] Call Trace:
<4> [111.929970] <TASK>
<4> [111.929974] ? lock_sync+0x100/0x100
<4> [111.929978] ? lock_release+0xd0/0x2b0
<4> [111.929983] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.929989] process_one_work+0x239/0x760
<4> [111.929995] worker_thread+0x200/0x3f0
<4> [111.929998] ? __pfx_worker_thread+0x10/0x10
<4> [111.930000] kthread+0x10d/0x150
<4> [111.930003] ? __pfx_kthread+0x10/0x10
<4> [111.930006] ret_from_fork+0x3d4/0x480
<4> [111.930008] ? __pfx_kthread+0x10/0x10
<4> [111.930012] ret_from_fork_asm+0x1a/0x30
<4> [111.930019] </TASK>
<4> [111.930020] irq event stamp: 179173
<4> [111.930021] hardirqs last enabled at (179179): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.930024] hardirqs last disabled at (179184): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.930026] softirqs last enabled at (178364): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.930029] softirqs last disabled at (178359): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.930031] ---[ end trace 0000000000000000 ]---
<5> [111.930392] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24127, lrc_seqno=24127, guc_id=0, flags=0x73 in no process [-1]
<7> [111.930469] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.930648] ------------[ cut here ]------------
<4> [111.930651] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.930654] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:17/6297
<4> [111.930777] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.930882] autofs4 [last unloaded: snd_hda_intel]
<4> [111.930888] CPU: 12 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.930892] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.930894] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.930896] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.930906] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.931034] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.931036] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.931040] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.931042] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.931044] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.931046] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.931048] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.931050] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [111.931052] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.931055] CR2: 00007b19c0095018 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [111.931057] PKRU: 55555554
<4> [111.931058] Call Trace:
<4> [111.931060] <TASK>
<4> [111.931067] ? lock_sync+0x100/0x100
<4> [111.931075] ? lock_release+0xd0/0x2b0
<4> [111.931084] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.931095] process_one_work+0x239/0x760
<4> [111.931105] worker_thread+0x200/0x3f0
<4> [111.931109] ? __pfx_worker_thread+0x10/0x10
<4> [111.931113] kthread+0x10d/0x150
<4> [111.931116] ? __pfx_kthread+0x10/0x10
<4> [111.931122] ret_from_fork+0x3d4/0x480
<4> [111.931125] ? __pfx_kthread+0x10/0x10
<4> [111.931130] ret_from_fork_asm+0x1a/0x30
<4> [111.931142] </TASK>
<4> [111.931144] irq event stamp: 840027
<4> [111.931146] hardirqs last enabled at (840033): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.931151] hardirqs last disabled at (840038): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.931154] softirqs last enabled at (838704): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [111.931157] softirqs last disabled at (838702): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [111.931160] ---[ end trace 0000000000000000 ]---
<6> [111.931164] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.931282] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.931291] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.931447] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.931553] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.931668] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.931923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.932059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.932198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.932331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.932464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.932595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.932734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.932871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.933002] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.933144] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.933310] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.934414] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.944722] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.944975] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.946247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.946369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.946488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.946610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.946749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.946871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.946991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.947112] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.947232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.947353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.947474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.947595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.947721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.947843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.947964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.948085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.948206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.948327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.948447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.948573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.948702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.948828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.948954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.949078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.949201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.949325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.949448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.949575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.949724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.949814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.949900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.949984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.950073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.950160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.950250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.950334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.950420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.950503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.950584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.950671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.950752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.950832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.950912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.950989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.951070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.951147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.951226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.951305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.951387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.951467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.951546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.951630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.951709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.951788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.951865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.951943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.952019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.952099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.952181] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.952295] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.952300] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24127, lrc_seqno=24127, guc_id=0, flags=0x73 in no process [-1]
<7> [111.952302] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.952361] ------------[ cut here ]------------
<4> [111.952362] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.952364] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [111.952435] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.952509] autofs4 [last unloaded: snd_hda_intel]
<4> [111.952513] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.952517] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.952518] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.952520] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.952526] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.952596] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.952598] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [111.952600] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.952602] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.952603] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.952605] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.952606] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.952608] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [111.952610] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.952611] CR2: 00007b19cb317000 CR3: 000000012654f003 CR4: 0000000000f72ef0
<4> [111.952613] PKRU: 55555554
<4> [111.952614] Call Trace:
<4> [111.952616] <TASK>
<4> [111.952620] ? lock_sync+0x100/0x100
<4> [111.952628] ? lock_release+0xd0/0x2b0
<4> [111.952634] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.952640] process_one_work+0x239/0x760
<4> [111.952646] worker_thread+0x200/0x3f0
<4> [111.952650] ? __pfx_worker_thread+0x10/0x10
<4> [111.952652] kthread+0x10d/0x150
<4> [111.952655] ? __pfx_kthread+0x10/0x10
<4> [111.952659] ret_from_fork+0x3d4/0x480
<4> [111.952661] ? __pfx_kthread+0x10/0x10
<4> [111.952665] ret_from_fork_asm+0x1a/0x30
<4> [111.952673] </TASK>
<4> [111.952674] irq event stamp: 843179
<4> [111.952675] hardirqs last enabled at (843185): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.952678] hardirqs last disabled at (843190): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.952680] softirqs last enabled at (843108): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.952683] softirqs last disabled at (843103): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.952685] ---[ end trace 0000000000000000 ]---
<6> [111.952687] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.952758] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.952764] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.954015] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.954133] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.954238] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.954448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.954543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.954646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.954740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.954829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.954918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.955005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.955093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.955176] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.955272] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.955391] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.956401] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.966626] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.966875] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.967895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.967974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.968052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.968133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.968213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.968290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.968366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.968442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.968518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.968592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.968681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.968756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.968831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.968908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.968986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.969064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.969139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.969214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.969289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.969373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.969460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.969541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.969630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.969710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.969789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.969868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.969950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.970033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.970115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.970195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.970276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.970356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.970440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.970525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.970608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.970702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.970785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.970867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.970947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.971026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.971111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.971194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.971274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.971352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.971436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.971518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.971599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.971692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.971772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.971849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.971926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.972005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.972083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.972159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.972234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.972312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.972388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.972464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.972544] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.972662] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.972666] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24127, lrc_seqno=24127, guc_id=0, flags=0x73 in no process [-1]
<7> [111.972669] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.972728] ------------[ cut here ]------------
<4> [111.972730] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.972731] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.972803] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.972870] autofs4 [last unloaded: snd_hda_intel]
<4> [111.972874] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.972877] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.972878] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.972880] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.972885] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.972956] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.972958] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.972960] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.972962] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.972963] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.972964] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.972965] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.972967] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.972968] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.972970] CR2: 00007b19cb319000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.972971] PKRU: 55555554
<4> [111.972972] Call Trace:
<4> [111.972973] <TASK>
<4> [111.972978] ? lock_sync+0x100/0x100
<4> [111.972982] ? lock_release+0xd0/0x2b0
<4> [111.972988] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.972993] process_one_work+0x239/0x760
<4> [111.973000] worker_thread+0x200/0x3f0
<4> [111.973003] ? __pfx_worker_thread+0x10/0x10
<4> [111.973005] kthread+0x10d/0x150
<4> [111.973008] ? __pfx_kthread+0x10/0x10
<4> [111.973012] ret_from_fork+0x3d4/0x480
<4> [111.973014] ? __pfx_kthread+0x10/0x10
<4> [111.973017] ret_from_fork_asm+0x1a/0x30
<4> [111.973024] </TASK>
<4> [111.973025] irq event stamp: 278245
<4> [111.973027] hardirqs last enabled at (278251): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.973030] hardirqs last disabled at (278256): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.973032] softirqs last enabled at (277142): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.973034] softirqs last disabled at (277135): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.973037] ---[ end trace 0000000000000000 ]---
<5> [111.974298] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24182, lrc_seqno=24182, guc_id=0, flags=0x73 in no process [-1]
<7> [111.974301] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.974364] ------------[ cut here ]------------
<4> [111.974365] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.974366] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [111.974437] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.974503] autofs4 [last unloaded: snd_hda_intel]
<4> [111.974507] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.974510] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.974511] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.974512] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.974517] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.974585] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.974587] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [111.974589] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.974590] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.974592] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.974593] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.974594] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.974596] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [111.974598] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.974599] CR2: 00007b19cb319000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [111.974601] PKRU: 55555554
<4> [111.974602] Call Trace:
<4> [111.974603] <TASK>
<4> [111.974607] ? lock_sync+0x100/0x100
<4> [111.974611] ? lock_release+0xd0/0x2b0
<4> [111.974622] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.974628] process_one_work+0x239/0x760
<4> [111.974634] worker_thread+0x200/0x3f0
<4> [111.974637] ? __pfx_worker_thread+0x10/0x10
<4> [111.974639] kthread+0x10d/0x150
<4> [111.974642] ? __pfx_kthread+0x10/0x10
<4> [111.974646] ret_from_fork+0x3d4/0x480
<4> [111.974648] ? __pfx_kthread+0x10/0x10
<4> [111.974652] ret_from_fork_asm+0x1a/0x30
<4> [111.974659] </TASK>
<4> [111.974661] irq event stamp: 280125
<4> [111.974662] hardirqs last enabled at (280131): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.974665] hardirqs last disabled at (280136): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.974667] softirqs last enabled at (280064): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.974670] softirqs last disabled at (280057): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.974672] ---[ end trace 0000000000000000 ]---
<7> [111.974687] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.974790] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [111.974886] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.974956] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.974962] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.975010] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.975437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.975532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.975635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.975724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.975810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.975896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.975981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.976069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.976156] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.976252] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.976369] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.977381] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [111.987622] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [111.987872] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [111.988911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [111.988991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [111.989068] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [111.989146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [111.989223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [111.989300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [111.989376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [111.989451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [111.989529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [111.989607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [111.989700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [111.989778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [111.989856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [111.989932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [111.990007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [111.990082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [111.990156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [111.990230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [111.990305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [111.990391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [111.990474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [111.990554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [111.990636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [111.990714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [111.990791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [111.990868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [111.990944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [111.991025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [111.991109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [111.991192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [111.991273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [111.991353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [111.991438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [111.991527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [111.991618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [111.991703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [111.991784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [111.991863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [111.991941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [111.992019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [111.992098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [111.992174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [111.992252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [111.992328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [111.992409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [111.992484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [111.992564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [111.992651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [111.992732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [111.992810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [111.992887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [111.992964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [111.993041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [111.993118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [111.993194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [111.993271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [111.993346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [111.993420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [111.993498] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [111.993608] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [111.994465] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24182, lrc_seqno=24182, guc_id=0, flags=0x73 in no process [-1]
<7> [111.994468] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [111.994531] ------------[ cut here ]------------
<4> [111.994533] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [111.994534] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [111.994605] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [111.994677] autofs4 [last unloaded: snd_hda_intel]
<4> [111.994681] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [111.994684] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [111.994685] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [111.994687] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [111.994692] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [111.994764] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [111.994766] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [111.994768] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [111.994770] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [111.994771] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [111.994772] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [111.994773] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [111.994775] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [111.994776] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [111.994778] CR2: 00007b19cb31c000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [111.994779] PKRU: 55555554
<4> [111.994780] Call Trace:
<4> [111.994782] <TASK>
<4> [111.994785] ? lock_sync+0x100/0x100
<4> [111.994790] ? lock_release+0xd0/0x2b0
<4> [111.994795] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [111.994801] process_one_work+0x239/0x760
<4> [111.994807] worker_thread+0x200/0x3f0
<4> [111.994810] ? __pfx_worker_thread+0x10/0x10
<4> [111.994812] kthread+0x10d/0x150
<4> [111.994815] ? __pfx_kthread+0x10/0x10
<4> [111.994819] ret_from_fork+0x3d4/0x480
<4> [111.994821] ? __pfx_kthread+0x10/0x10
<4> [111.994825] ret_from_fork_asm+0x1a/0x30
<4> [111.994832] </TASK>
<4> [111.994833] irq event stamp: 185521
<4> [111.994834] hardirqs last enabled at (185527): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [111.994837] hardirqs last disabled at (185532): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [111.994839] softirqs last enabled at (184834): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.994842] softirqs last disabled at (184829): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [111.994844] ---[ end trace 0000000000000000 ]---
<6> [111.994846] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [111.994917] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [111.994927] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [111.995469] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [111.995575] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [111.995725] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [111.995928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [111.996017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [111.996111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [111.996203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [111.996292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [111.996377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [111.996463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [111.996550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [111.996637] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [111.996732] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [111.996852] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [111.997859] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.007622] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.007869] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.008970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.009048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.009125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.009203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.009279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.009356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.009435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.009512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.009589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.009674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.009749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.009824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.009898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.009971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.010045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.010119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.010194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.010267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.010341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.010425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.010509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.010588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.010678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.010756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.010834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.010912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.010989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.011072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.011155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.011235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.011316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.011396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.011484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.011569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.011663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.011744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.011825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.011903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.011982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.012059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.012139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.012219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.012297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.012373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.012453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.012531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.012608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.012699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.012782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.012861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.012939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.013019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.013097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.013175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.013252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.013330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.013406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.013483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.013564] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.013684] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.013688] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24182, lrc_seqno=24182, guc_id=0, flags=0x73 in no process [-1]
<7> [112.013691] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.013750] ------------[ cut here ]------------
<4> [112.013751] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.013753] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.013826] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.013897] autofs4 [last unloaded: snd_hda_intel]
<4> [112.013901] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.013904] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.013905] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.013906] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.013911] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.013981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.013983] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.013985] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.013987] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.013988] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.013989] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.013990] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.013992] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.013993] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.013994] CR2: 00007b19cb31c000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [112.013996] PKRU: 55555554
<4> [112.013997] Call Trace:
<4> [112.013998] <TASK>
<4> [112.014002] ? lock_sync+0x100/0x100
<4> [112.014006] ? lock_release+0xd0/0x2b0
<4> [112.014012] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.014017] process_one_work+0x239/0x760
<4> [112.014024] worker_thread+0x200/0x3f0
<4> [112.014026] ? __pfx_worker_thread+0x10/0x10
<4> [112.014029] kthread+0x10d/0x150
<4> [112.014032] ? __pfx_kthread+0x10/0x10
<4> [112.014035] ret_from_fork+0x3d4/0x480
<4> [112.014037] ? __pfx_kthread+0x10/0x10
<4> [112.014041] ret_from_fork_asm+0x1a/0x30
<4> [112.014048] </TASK>
<4> [112.014049] irq event stamp: 188641
<4> [112.014050] hardirqs last enabled at (188647): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.014053] hardirqs last disabled at (188652): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.014055] softirqs last enabled at (187628): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.014058] softirqs last disabled at (187621): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.014060] ---[ end trace 0000000000000000 ]---
<5> [112.014412] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24237, lrc_seqno=24237, guc_id=0, flags=0x73 in no process [-1]
<7> [112.014420] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.014681] ------------[ cut here ]------------
<4> [112.014684] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.014688] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:17/6297
<4> [112.014810] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.014925] autofs4 [last unloaded: snd_hda_intel]
<4> [112.014932] CPU: 12 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.014936] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.014939] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.014941] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.014951] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.015070] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.015072] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.015076] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.015078] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.015080] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.015082] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.015084] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.015086] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [112.015088] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.015091] CR2: 00007b19c0095018 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.015093] PKRU: 55555554
<4> [112.015094] Call Trace:
<4> [112.015096] <TASK>
<4> [112.015103] ? lock_sync+0x100/0x100
<4> [112.015112] ? lock_release+0xd0/0x2b0
<4> [112.015121] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.015130] process_one_work+0x239/0x760
<4> [112.015141] worker_thread+0x200/0x3f0
<4> [112.015146] ? __pfx_worker_thread+0x10/0x10
<4> [112.015150] kthread+0x10d/0x150
<4> [112.015154] ? __pfx_kthread+0x10/0x10
<4> [112.015159] ret_from_fork+0x3d4/0x480
<4> [112.015163] ? __pfx_kthread+0x10/0x10
<4> [112.015167] ret_from_fork_asm+0x1a/0x30
<4> [112.015180] </TASK>
<4> [112.015183] irq event stamp: 845891
<4> [112.015185] hardirqs last enabled at (845897): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.015189] hardirqs last disabled at (845902): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.015193] softirqs last enabled at (845130): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.015196] softirqs last disabled at (845125): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.015200] ---[ end trace 0000000000000000 ]---
<6> [112.015203] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.015323] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.015332] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.015387] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.015491] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.015597] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.015816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.015904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.015997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.016086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.016174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.016263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.016355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.016446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.016532] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.016630] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.016748] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.017757] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.027614] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.027862] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.028888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.028968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.029045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.029122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.029199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.029275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.029351] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.029426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.029502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.029577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.029660] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.029735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.029810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.029886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.029961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.030035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.030110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.030185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.030262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.030348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.030431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.030512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.030592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.030705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.030783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.030860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.030938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.031018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.031102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.031183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.031264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.031345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.031430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.031516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.031601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.031696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.031776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.031853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.031930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.032006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.032086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.032165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.032246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.032325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.032409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.032487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.032566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.032652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.032734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.032812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.032890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.032968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.033045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.033123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.033200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.033278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.033353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.033431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.033514] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.033629] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.033633] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24237, lrc_seqno=24237, guc_id=0, flags=0x73 in no process [-1]
<7> [112.033635] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.033694] ------------[ cut here ]------------
<4> [112.033695] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.033696] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.033767] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.033837] autofs4 [last unloaded: snd_hda_intel]
<4> [112.033842] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.033844] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.033846] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.033847] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.033853] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.033923] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.033925] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.033927] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.033929] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.033930] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.033932] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.033933] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.033935] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.033936] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.033938] CR2: 00007b19cb31c000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [112.033940] PKRU: 55555554
<4> [112.033941] Call Trace:
<4> [112.033942] <TASK>
<4> [112.033946] ? lock_sync+0x100/0x100
<4> [112.033951] ? lock_release+0xd0/0x2b0
<4> [112.033957] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.033963] process_one_work+0x239/0x760
<4> [112.033969] worker_thread+0x200/0x3f0
<4> [112.033972] ? __pfx_worker_thread+0x10/0x10
<4> [112.033974] kthread+0x10d/0x150
<4> [112.033977] ? __pfx_kthread+0x10/0x10
<4> [112.033981] ret_from_fork+0x3d4/0x480
<4> [112.033983] ? __pfx_kthread+0x10/0x10
<4> [112.033987] ret_from_fork_asm+0x1a/0x30
<4> [112.033994] </TASK>
<4> [112.033996] irq event stamp: 193051
<4> [112.033997] hardirqs last enabled at (193057): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.034000] hardirqs last disabled at (193062): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.034003] softirqs last enabled at (192048): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.034005] softirqs last disabled at (192041): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.034008] ---[ end trace 0000000000000000 ]---
<6> [112.034009] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.034081] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.034087] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.034490] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.034698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.034787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.034880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.034969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.035058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.035144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.035232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.035324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.035408] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.035505] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.035625] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.036634] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.047356] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.047608] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.048806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.048885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.048961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.049038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.049115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.049191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.049267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.049342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.049418] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.049492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.049571] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.049659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.049735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.049812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.049887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.049960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.050033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.050106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.050180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.050261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.050343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.050423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.050503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.050581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.050670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.050752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.050833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.050917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.051002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.051083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.051165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.051245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.051331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.051416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.051501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.051582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.051670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.051749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.051827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.051903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.051986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.052067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.052146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.052224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.052306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.052384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.052462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.052539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.052618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.052695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.052770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.052850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.052928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.053006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.053085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.053166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.053245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.053322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.053401] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.053510] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.053514] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24237, lrc_seqno=24237, guc_id=0, flags=0x73 in no process [-1]
<7> [112.053517] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.053575] ------------[ cut here ]------------
<4> [112.053577] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.053578] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.053659] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.053724] autofs4 [last unloaded: snd_hda_intel]
<4> [112.053728] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.053731] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.053732] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.053734] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.053739] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.053808] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.053810] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.053812] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.053813] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.053815] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.053816] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.053817] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.053818] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.053820] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.053821] CR2: 00007b19cb31c000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.053823] PKRU: 55555554
<4> [112.053824] Call Trace:
<4> [112.053825] <TASK>
<4> [112.053829] ? lock_sync+0x100/0x100
<4> [112.053833] ? lock_release+0xd0/0x2b0
<4> [112.053838] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.053844] process_one_work+0x239/0x760
<4> [112.053850] worker_thread+0x200/0x3f0
<4> [112.053853] ? __pfx_worker_thread+0x10/0x10
<4> [112.053855] kthread+0x10d/0x150
<4> [112.053858] ? __pfx_kthread+0x10/0x10
<4> [112.053862] ret_from_fork+0x3d4/0x480
<4> [112.053863] ? __pfx_kthread+0x10/0x10
<4> [112.053867] ret_from_fork_asm+0x1a/0x30
<4> [112.053874] </TASK>
<4> [112.053875] irq event stamp: 196141
<4> [112.053876] hardirqs last enabled at (196147): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.053879] hardirqs last disabled at (196152): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.053881] softirqs last enabled at (195398): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.053883] softirqs last disabled at (195393): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.053886] ---[ end trace 0000000000000000 ]---
<5> [112.055195] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24292, lrc_seqno=24292, guc_id=0, flags=0x73 in no process [-1]
<7> [112.055199] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [112.055205] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [112.055265] ------------[ cut here ]------------
<4> [112.055266] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.055267] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [112.055342] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core
<7> [112.055310] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.055359] drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [112.055416] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.055419] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.055420] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.055422] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.055427] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.055500] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.055501] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.055504] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.055505] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.055506] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.055508] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.055509] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.055510] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.055512] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.055514] CR2: 00007b19cb31d000 CR3: 000000012654f002 CR4: 0000000000f72ef0
<4> [112.055515] PKRU: 55555554
<4> [112.055516] Call Trace:
<4> [112.055517] <TASK>
<4> [112.055521] ? lock_sync+0x100/0x100
<4> [112.055526] ? lock_release+0xd0/0x2b0
<4> [112.055531] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.055537] process_one_work+0x239/0x760
<4> [112.055543] worker_thread+0x200/0x3f0
<4> [112.055546] ? __pfx_worker_thread+0x10/0x10
<4> [112.055548] kthread+0x10d/0x150
<4> [112.055551] ? __pfx_kthread+0x10/0x10
<4> [112.055555] ret_from_fork+0x3d4/0x480
<4> [112.055557] ? __pfx_kthread+0x10/0x10
<4> [112.055560] ret_from_fork_asm+0x1a/0x30
<4> [112.055567] </TASK>
<4> [112.055569] irq event stamp: 282967
<4> [112.055570] hardirqs last enabled at (282973): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.055573] hardirqs last disabled at (282978): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.055576] softirqs last enabled at (281966): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [112.055578] softirqs last disabled at (281964): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [112.055581] ---[ end trace 0000000000000000 ]---
<6> [112.055583] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.055669] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.055677] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.055807] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.056020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.056113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.056211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.056301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.056393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.056481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.056568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.056666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.056749] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.056846] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.056963] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.057972] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.068605] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.068855] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.069879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.069959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.070036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.070113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.070189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.070265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.070341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.070417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.070493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.070568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.070662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.070737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.070814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.070893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.070973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.071050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.071128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.071206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.071283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.071366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.071448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.071531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.071619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.071699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.071778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.071855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.071933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.072014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.072095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.072174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.072254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.072333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.072418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.072502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.072587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.072681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.072762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.072841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.072920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.072998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.073083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.073168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.073248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.073327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.073408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.073486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.073563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.073648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.073727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.073803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.073880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.073959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.074036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.074114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.074190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.074267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.074346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.074426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.074508] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.074625] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.074629] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24292, lrc_seqno=24292, guc_id=0, flags=0x73 in no process [-1]
<7> [112.074632] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.074690] ------------[ cut here ]------------
<4> [112.074691] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.074693] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.074764] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.074829] autofs4 [last unloaded: snd_hda_intel]
<4> [112.074833] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.074836] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.074838] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.074839] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.074845] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.074913] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.074915] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.074917] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.074919] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.074920] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.074922] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.074923] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.074924] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.074926] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.074927] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.074929] PKRU: 55555554
<4> [112.074930] Call Trace:
<4> [112.074931] <TASK>
<4> [112.074935] ? lock_sync+0x100/0x100
<4> [112.074940] ? lock_release+0xd0/0x2b0
<4> [112.074945] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.074951] process_one_work+0x239/0x760
<4> [112.074957] worker_thread+0x200/0x3f0
<4> [112.074960] ? __pfx_worker_thread+0x10/0x10
<4> [112.074962] kthread+0x10d/0x150
<4> [112.074965] ? __pfx_kthread+0x10/0x10
<4> [112.074969] ret_from_fork+0x3d4/0x480
<4> [112.074971] ? __pfx_kthread+0x10/0x10
<4> [112.074974] ret_from_fork_asm+0x1a/0x30
<4> [112.074982] </TASK>
<4> [112.074983] irq event stamp: 174119
<4> [112.074984] hardirqs last enabled at (174125): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.074987] hardirqs last disabled at (174130): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.074989] softirqs last enabled at (173016): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.074992] softirqs last disabled at (173009): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.074994] ---[ end trace 0000000000000000 ]---
<6> [112.074996] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.075066] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.075072] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.076315] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.076421] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.076529] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.076737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.076828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.076923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.077013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.077103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.077192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.077284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.077374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.077458] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.077555] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.077681] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.078688] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.089410] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.089660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.090691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.090771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.090850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.090927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.091005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.091082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.091159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.091239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.091318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.091394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.091470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.091545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.091627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.091704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.091779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.091853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.091927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.092001] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.092079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.092162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.092244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.092323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.092402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.092481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.092560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.092646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.092725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.092805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.092886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.092966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.093045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.093125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.093210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.093294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.093380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.093467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.093551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.093639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.093718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.093795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.093875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.093955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.094036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.094115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.094197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.094275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.094354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.094430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.094509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.094585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.094674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.094753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.094835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.094917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.094995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.095075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.095151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.095229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.095309] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.095420] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.095424] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24292, lrc_seqno=24292, guc_id=0, flags=0x73 in no process [-1]
<7> [112.095426] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.095485] ------------[ cut here ]------------
<4> [112.095486] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.095487] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.095557] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.095633] autofs4 [last unloaded: snd_hda_intel]
<4> [112.095638] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.095641] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.095642] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.095644] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.095649] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.095719] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.095720] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.095723] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.095725] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.095726] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.095728] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.095729] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.095731] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.095732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.095734] CR2: 00007b19cb31c000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.095736] PKRU: 55555554
<4> [112.095737] Call Trace:
<4> [112.095738] <TASK>
<4> [112.095743] ? lock_sync+0x100/0x100
<4> [112.095747] ? lock_release+0xd0/0x2b0
<4> [112.095753] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.095759] process_one_work+0x239/0x760
<4> [112.095765] worker_thread+0x200/0x3f0
<4> [112.095768] ? __pfx_worker_thread+0x10/0x10
<4> [112.095771] kthread+0x10d/0x150
<4> [112.095774] ? __pfx_kthread+0x10/0x10
<4> [112.095778] ret_from_fork+0x3d4/0x480
<4> [112.095780] ? __pfx_kthread+0x10/0x10
<4> [112.095783] ret_from_fork_asm+0x1a/0x30
<4> [112.095791] </TASK>
<4> [112.095792] irq event stamp: 200397
<4> [112.095794] hardirqs last enabled at (200403): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.095797] hardirqs last disabled at (200408): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.095799] softirqs last enabled at (199410): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.095802] softirqs last disabled at (199403): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.095804] ---[ end trace 0000000000000000 ]---
<7> [112.097090] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.097198] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [112.097298] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24347, lrc_seqno=24347, guc_id=0, flags=0x73 in no process [-1]
<7> [112.097301] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.097360] ------------[ cut here ]------------
<4> [112.097361] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.097363] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.097435] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.097498] autofs4 [last unloaded: snd_hda_intel]
<4> [112.097502] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.097505] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.097506] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.097507] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.097512] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.097583] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.097584] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.097586] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.097588] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.097589] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.097590] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.097599] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.097601] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.097602] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.097604] CR2: 00007b19cb31c000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.097605] PKRU: 55555554
<4> [112.097606] Call Trace:
<4> [112.097607] <TASK>
<4> [112.097611] ? lock_sync+0x100/0x100
<4> [112.097615] ? lock_release+0xd0/0x2b0
<4> [112.097621] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.097627] process_one_work+0x239/0x760
<4> [112.097633] worker_thread+0x200/0x3f0
<4> [112.097636] ? __pfx_worker_thread+0x10/0x10
<4> [112.097639] kthread+0x10d/0x150
<4> [112.097642] ? __pfx_kthread+0x10/0x10
<4> [112.097645] ret_from_fork+0x3d4/0x480
<4> [112.097647] ? __pfx_kthread+0x10/0x10
<4> [112.097651] ret_from_fork_asm+0x1a/0x30
<4> [112.097659] </TASK>
<4> [112.097660] irq event stamp: 202283
<4> [112.097661] hardirqs last enabled at (202289): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.097664] hardirqs last disabled at (202294): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.097666] softirqs last enabled at (202166): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.097669] softirqs last disabled at (202155): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.097671] ---[ end trace 0000000000000000 ]---
<6> [112.097673] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.097743] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.097750] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.097987] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.098194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.098285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.098379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.098468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.098558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.098658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.098748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.098838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.098921] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.099017] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.099135] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.100142] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.110597] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.110848] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.111856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.111934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.112012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.112090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.112167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.112243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.112319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.112395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.112470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.112546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.112630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.112705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.112779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.112853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.112926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.113005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.113082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.113158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.113234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.113317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.113401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.113480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.113560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.113652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.113732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.113814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.113893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.113973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.114055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.114135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.114215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.114294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.114377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.114462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.114545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.114635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.114715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.114792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.114869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.114946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.115026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.115104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.115185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.115266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.115349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.115427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.115505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.115582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.115673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.115748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.115824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.115903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.115981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.116059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.116135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.116212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.116292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.116373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.116455] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.116567] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.116571] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24347, lrc_seqno=24347, guc_id=0, flags=0x73 in no process [-1]
<7> [112.116574] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.116653] ------------[ cut here ]------------
<4> [112.116654] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.116656] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.116728] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.116792] autofs4 [last unloaded: snd_hda_intel]
<4> [112.116796] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.116799] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.116800] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.116801] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.116807] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.116876] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.116878] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.116880] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.116882] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.116883] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.116884] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.116886] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.116887] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.116888] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.116890] CR2: 00007b19cb31b000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [112.116891] PKRU: 55555554
<4> [112.116893] Call Trace:
<4> [112.116894] <TASK>
<4> [112.116897] ? lock_sync+0x100/0x100
<4> [112.116902] ? lock_release+0xd0/0x2b0
<4> [112.116907] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.116913] process_one_work+0x239/0x760
<4> [112.116919] worker_thread+0x200/0x3f0
<4> [112.116922] ? __pfx_worker_thread+0x10/0x10
<4> [112.116924] kthread+0x10d/0x150
<4> [112.116927] ? __pfx_kthread+0x10/0x10
<4> [112.116931] ret_from_fork+0x3d4/0x480
<4> [112.116933] ? __pfx_kthread+0x10/0x10
<4> [112.116937] ret_from_fork_asm+0x1a/0x30
<4> [112.116944] </TASK>
<4> [112.116945] irq event stamp: 177711
<4> [112.116946] hardirqs last enabled at (177717): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.116949] hardirqs last disabled at (177722): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.116951] softirqs last enabled at (176950): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.116954] softirqs last disabled at (176939): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.116956] ---[ end trace 0000000000000000 ]---
<6> [112.116958] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.117029] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.117035] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.118271] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.118376] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.118481] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.118692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.118784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.118880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.118969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.119059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.119149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.119239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.119327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.119411] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.119505] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.119626] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.120638] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.131359] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.131618] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.132632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.132718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.132797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.132876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.132956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.133034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.133112] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.133189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.133265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.133343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.133421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.133497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.133573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.133661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.133736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.133812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.133887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.133963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.134041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.134124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.134206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.134285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.134365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.134442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.134519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.134599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.134675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.134755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.134839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.134921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.135001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.135081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.135167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.135252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.135337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.135418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.135498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.135578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.135676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.135755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.135834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.135912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.135989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.136065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.136145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.136222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.136299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.136375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.136453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.136534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.136623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.136704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.136782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.136861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.136941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.137023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.137101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.137180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.137262] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.137375] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.137379] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24347, lrc_seqno=24347, guc_id=0, flags=0x73 in no process [-1]
<7> [112.137381] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.137441] ------------[ cut here ]------------
<4> [112.137442] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.137443] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [112.137514] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.137590] autofs4 [last unloaded: snd_hda_intel]
<4> [112.137595] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.137599] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.137600] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.137601] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.137607] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.137677] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.137679] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.137681] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.137683] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.137684] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.137686] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.137687] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.137689] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.137691] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.137692] CR2: 00007b19cb31d000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.137694] PKRU: 55555554
<4> [112.137695] Call Trace:
<4> [112.137697] <TASK>
<4> [112.137701] ? lock_sync+0x100/0x100
<4> [112.137706] ? lock_release+0xd0/0x2b0
<4> [112.137711] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.137717] process_one_work+0x239/0x760
<4> [112.137724] worker_thread+0x200/0x3f0
<4> [112.137727] ? __pfx_worker_thread+0x10/0x10
<4> [112.137729] kthread+0x10d/0x150
<4> [112.137732] ? __pfx_kthread+0x10/0x10
<4> [112.137736] ret_from_fork+0x3d4/0x480
<4> [112.137738] ? __pfx_kthread+0x10/0x10
<4> [112.137742] ret_from_fork_asm+0x1a/0x30
<4> [112.137750] </TASK>
<4> [112.137751] irq event stamp: 288289
<4> [112.137752] hardirqs last enabled at (288295): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.137755] hardirqs last disabled at (288300): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.137758] softirqs last enabled at (288074): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.137761] softirqs last disabled at (288069): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.137763] ---[ end trace 0000000000000000 ]---
<5> [112.138371] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24402, lrc_seqno=24402, guc_id=0, flags=0x73 in no process [-1]
<7> [112.138376] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.138477] ------------[ cut here ]------------
<4> [112.138478] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.138480] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.138552] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.138650] autofs4 [last unloaded: snd_hda_intel]
<4> [112.138654] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.138657] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.138658] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.138660] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.138667] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.138759] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.138761] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.138765] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.138766] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.138767] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.138768] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.138770] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.138771] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.138773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.138774] CR2: 00007b19cb31b000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [112.138776] PKRU: 55555554
<4> [112.138777] Call Trace:
<4> [112.138778] <TASK>
<4> [112.138782] ? lock_sync+0x100/0x100
<4> [112.138788] ? lock_release+0xd0/0x2b0
<4> [112.138794] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.138800] process_one_work+0x239/0x760
<4> [112.138807] worker_thread+0x200/0x3f0
<4> [112.138809] ? __pfx_worker_thread+0x10/0x10
<4> [112.138812] kthread+0x10d/0x150
<4> [112.138815] ? __pfx_kthread+0x10/0x10
<4> [112.138819] ret_from_fork+0x3d4/0x480
<4> [112.138822] ? __pfx_kthread+0x10/0x10
<4> [112.138825] ret_from_fork_asm+0x1a/0x30
<4> [112.138832] </TASK>
<4> [112.138834] irq event stamp: 179245
<4> [112.138835] hardirqs last enabled at (179251): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.138839] hardirqs last disabled at (179256): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.138842] softirqs last enabled at (178682): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.138846] softirqs last disabled at (178623): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.138849] ---[ end trace 0000000000000000 ]---
<6> [112.138851] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.138937] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.138948] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.139253] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.139358] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.139462] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.139669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.139759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.139852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.139941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.140028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.140116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.140203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.140291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.140377] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.140474] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.140599] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.141608] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.151590] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.151840] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.152866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.152946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.153023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.153100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.153175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.153251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.153326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.153400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.153474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.153548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.153634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.153712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.153789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.153863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.153937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.154012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.154086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.154161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.154236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.154318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.154398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.154477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.154559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.154649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.154727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.154806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.154883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.154963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.155045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.155128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.155211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.155291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.155375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.155459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.155544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.155631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.155710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.155787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.155864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.155939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.156017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.156093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.156169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.156245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.156327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.156407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.156484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.156565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.156656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.156735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.156812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.156891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.156969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.157046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.157123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.157200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.157275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.157354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.157436] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.157548] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.157552] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24402, lrc_seqno=24402, guc_id=0, flags=0x73 in no process [-1]
<7> [112.157555] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.157630] ------------[ cut here ]------------
<4> [112.157632] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.157633] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.157704] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.157769] autofs4 [last unloaded: snd_hda_intel]
<4> [112.157773] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.157775] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.157776] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.157778] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.157783] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.157852] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.157854] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.157856] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.157857] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.157859] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.157860] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.157861] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.157862] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.157864] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.157865] CR2: 00007b19cb31b000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [112.157867] PKRU: 55555554
<4> [112.157868] Call Trace:
<4> [112.157869] <TASK>
<4> [112.157873] ? lock_sync+0x100/0x100
<4> [112.157877] ? lock_release+0xd0/0x2b0
<4> [112.157882] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.157888] process_one_work+0x239/0x760
<4> [112.157894] worker_thread+0x200/0x3f0
<4> [112.157897] ? __pfx_worker_thread+0x10/0x10
<4> [112.157899] kthread+0x10d/0x150
<4> [112.157902] ? __pfx_kthread+0x10/0x10
<4> [112.157906] ret_from_fork+0x3d4/0x480
<4> [112.157908] ? __pfx_kthread+0x10/0x10
<4> [112.157911] ret_from_fork_asm+0x1a/0x30
<4> [112.157919] </TASK>
<4> [112.157920] irq event stamp: 182387
<4> [112.157921] hardirqs last enabled at (182393): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.157924] hardirqs last disabled at (182398): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.157926] softirqs last enabled at (181626): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.157929] softirqs last disabled at (181603): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.157931] ---[ end trace 0000000000000000 ]---
<6> [112.157933] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.158003] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.158008] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.158416] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.158629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.158714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.158804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.158889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.158975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.159061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.159149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.159237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.159319] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.159414] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.159530] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.160538] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.171269] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.171509] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.172507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.172582] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.172658] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.172731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.172803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.172873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.172942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.173010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.173078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.173147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.173215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.173282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.173350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.173419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.173486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.173554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.173633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.173707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.173780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.173862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.173943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.174024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.174104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.174182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.174257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.174335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.174412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.174491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.174572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.174664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.174746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.174829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.174916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.175002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.175086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.175167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.175247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.175324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.175403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.175479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.175558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.175645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.175724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.175803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.175887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.175966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.176045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.176123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.176201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.176277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.176353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.176436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.176518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.176600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.176679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.176758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.176834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.176913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.176997] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.177107] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.177111] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24402, lrc_seqno=24402, guc_id=0, flags=0x73 in no process [-1]
<7> [112.177114] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.177172] ------------[ cut here ]------------
<4> [112.177173] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.177174] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.177245] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.177308] autofs4 [last unloaded: snd_hda_intel]
<4> [112.177312] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.177315] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.177316] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.177317] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.177322] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.177392] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.177393] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.177395] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.177397] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.177398] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.177399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.177400] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.177402] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.177403] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.177404] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.177406] PKRU: 55555554
<4> [112.177407] Call Trace:
<4> [112.177408] <TASK>
<4> [112.177412] ? lock_sync+0x100/0x100
<4> [112.177416] ? lock_release+0xd0/0x2b0
<4> [112.177421] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.177427] process_one_work+0x239/0x760
<4> [112.177433] worker_thread+0x200/0x3f0
<4> [112.177436] ? __pfx_worker_thread+0x10/0x10
<4> [112.177438] kthread+0x10d/0x150
<4> [112.177441] ? __pfx_kthread+0x10/0x10
<4> [112.177445] ret_from_fork+0x3d4/0x480
<4> [112.177447] ? __pfx_kthread+0x10/0x10
<4> [112.177450] ret_from_fork_asm+0x1a/0x30
<4> [112.177457] </TASK>
<4> [112.177458] irq event stamp: 185545
<4> [112.177459] hardirqs last enabled at (185551): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.177462] hardirqs last disabled at (185556): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.177464] softirqs last enabled at (184488): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.177467] softirqs last disabled at (184481): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.177469] ---[ end trace 0000000000000000 ]---
<7> [112.178823] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<5> [112.178975] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24457, lrc_seqno=24457, guc_id=0, flags=0x73 in no process [-1]
<7> [112.178930] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.178977] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.179037] ------------[ cut here ]------------
<4> [112.179038] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.179040] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<4> [112.179113] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.179179] autofs4 [last unloaded: snd_hda_intel]
<4> [112.179184] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.179187] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.179188] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.179190] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.179195] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.179265] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.179267] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.179269] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.179271] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.179272] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.179274] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.179275] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.179277] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.179278] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.179280] CR2: 00007b19cb31d000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.179282] PKRU: 55555554
<4> [112.179283] Call Trace:
<4> [112.179284] <TASK>
<4> [112.179288] ? lock_sync+0x100/0x100
<4> [112.179293] ? lock_release+0xd0/0x2b0
<4> [112.179298] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.179304] process_one_work+0x239/0x760
<4> [112.179310] worker_thread+0x200/0x3f0
<4> [112.179313] ? __pfx_worker_thread+0x10/0x10
<4> [112.179315] kthread+0x10d/0x150
<4> [112.179318] ? __pfx_kthread+0x10/0x10
<4> [112.179322] ret_from_fork+0x3d4/0x480
<4> [112.179324] ? __pfx_kthread+0x10/0x10
<4> [112.179327] ret_from_fork_asm+0x1a/0x30
<4> [112.179334] </TASK>
<4> [112.179335] irq event stamp: 290719
<4> [112.179337] hardirqs last enabled at (290725): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.179339] hardirqs last disabled at (290730): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.179341] softirqs last enabled at (289222): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [112.179344] softirqs last disabled at (289220): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [112.179346] ---[ end trace 0000000000000000 ]---
<6> [112.179348] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.179417] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.179423] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.179469] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.179674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.179763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.179857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.179945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.180032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.180118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.180204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.180292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.180375] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.180470] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.180590] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.181597] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.191581] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.191829] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.192795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.192876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.192954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.193031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.193108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.193184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.193260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.193335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.193412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.193488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.193565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.193659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.193735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.193812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.193889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.193965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.194040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.194116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.194191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.194273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.194354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.194434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.194514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.194597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.194675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.194756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.194836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.194916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.194998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.195076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.195154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.195235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.195324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.195410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.195495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.195581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.195663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.195740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.195817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.195894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.195973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.196048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.196128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.196209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.196291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.196369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.196449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.196531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.196622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.196701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.196778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.196858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.196936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.197013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.197088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.197166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.197244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.197321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.197402] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.197511] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.197515] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24457, lrc_seqno=24457, guc_id=0, flags=0x73 in no process [-1]
<7> [112.197517] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.199135] ------------[ cut here ]------------
<4> [112.199136] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.199138] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.199216] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.199282] autofs4 [last unloaded: snd_hda_intel]
<4> [112.199286] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.199289] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.199290] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.199292] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.199297] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.199368] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.199370] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.199372] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.199373] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.199375] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.199376] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.199377] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.199378] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.199380] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.199381] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.199383] PKRU: 55555554
<4> [112.199384] Call Trace:
<4> [112.199385] <TASK>
<4> [112.199389] ? lock_sync+0x100/0x100
<4> [112.199393] ? lock_release+0xd0/0x2b0
<4> [112.199399] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.199404] process_one_work+0x239/0x760
<4> [112.199411] worker_thread+0x200/0x3f0
<4> [112.199413] ? __pfx_worker_thread+0x10/0x10
<4> [112.199416] kthread+0x10d/0x150
<4> [112.199419] ? __pfx_kthread+0x10/0x10
<4> [112.199422] ret_from_fork+0x3d4/0x480
<4> [112.199424] ? __pfx_kthread+0x10/0x10
<4> [112.199428] ret_from_fork_asm+0x1a/0x30
<4> [112.199435] </TASK>
<4> [112.199436] irq event stamp: 189085
<4> [112.199437] hardirqs last enabled at (189091): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.199440] hardirqs last disabled at (189096): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.199442] softirqs last enabled at (188172): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.199445] softirqs last disabled at (188165): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.199447] ---[ end trace 0000000000000000 ]---
<6> [112.199449] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.199520] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.199527] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.199541] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.199659] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.200089] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.200297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.200390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.200487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.200583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.200676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.200765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.200852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.200939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.201022] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.201117] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.201235] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.202241] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.212577] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.212825] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.213828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.213907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.213983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.214061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.214138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.214214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.214290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.214365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.214441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.214521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.214606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.214682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.214757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.214832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.214907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.214981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.215055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.215130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.215204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.215286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.215367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.215446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.215529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.215621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.215699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.215776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.215852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.215930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.216011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.216094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.216174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.216255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.216340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.216425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.216510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.216598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.216683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.216768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.216850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.216931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.217012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.217086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.217158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.217230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.217304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.217377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.217452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.217525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.217607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.217685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.217758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.217831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.217901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.217971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.218040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.218111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.218181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.218249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.218322] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.218427] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.218431] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24457, lrc_seqno=24457, guc_id=0, flags=0x73 in no process [-1]
<7> [112.218434] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.218492] ------------[ cut here ]------------
<4> [112.218493] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.218495] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.218560] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.218625] autofs4 [last unloaded: snd_hda_intel]
<4> [112.218630] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.218633] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.218634] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.218635] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.218641] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.218713] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.218715] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.218717] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.218718] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.218719] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.218720] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.218721] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.218723] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.218724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.218725] CR2: 00007b19cb31f000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [112.218727] PKRU: 55555554
<4> [112.218728] Call Trace:
<4> [112.218729] <TASK>
<4> [112.218732] ? lock_sync+0x100/0x100
<4> [112.218736] ? lock_release+0xd0/0x2b0
<4> [112.218741] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.218746] process_one_work+0x239/0x760
<4> [112.218752] worker_thread+0x200/0x3f0
<4> [112.218755] ? __pfx_worker_thread+0x10/0x10
<4> [112.218757] kthread+0x10d/0x150
<4> [112.218760] ? __pfx_kthread+0x10/0x10
<4> [112.218763] ret_from_fork+0x3d4/0x480
<4> [112.218765] ? __pfx_kthread+0x10/0x10
<4> [112.218768] ret_from_fork_asm+0x1a/0x30
<4> [112.218775] </TASK>
<4> [112.218776] irq event stamp: 211027
<4> [112.218777] hardirqs last enabled at (211033): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.218780] hardirqs last disabled at (211038): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.218782] softirqs last enabled at (210118): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.218784] softirqs last disabled at (210111): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.218786] ---[ end trace 0000000000000000 ]---
<5> [112.220076] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24512, lrc_seqno=24512, guc_id=0, flags=0x73 in no process [-1]
<7> [112.220082] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [112.220084] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [112.220158] ------------[ cut here ]------------
<4> [112.220159] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.220161] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:157/6437
<7> [112.220190] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.220241] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.220321] autofs4 [last unloaded: snd_hda_intel]
<4> [112.220325] CPU: 0 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.220328] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.220330] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.220331] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.220337] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.220412] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.220414] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.220416] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.220418] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.220419] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.220420] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.220422] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.220423] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.220425] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.220426] CR2: 00007b19cb31d000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.220427] PKRU: 55555554
<4> [112.220429] Call Trace:
<4> [112.220430] <TASK>
<4> [112.220434] ? lock_sync+0x100/0x100
<4> [112.220439] ? lock_release+0xd0/0x2b0
<4> [112.220445] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.220450] process_one_work+0x239/0x760
<4> [112.220457] worker_thread+0x200/0x3f0
<4> [112.220460] ? __pfx_worker_thread+0x10/0x10
<4> [112.220462] kthread+0x10d/0x150
<4> [112.220465] ? __pfx_kthread+0x10/0x10
<4> [112.220469] ret_from_fork+0x3d4/0x480
<4> [112.220471] ? __pfx_kthread+0x10/0x10
<4> [112.220474] ret_from_fork_asm+0x1a/0x30
<4> [112.220482] </TASK>
<4> [112.220483] irq event stamp: 292663
<4> [112.220484] hardirqs last enabled at (292669): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.220487] hardirqs last disabled at (292674): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.220490] softirqs last enabled at (291350): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [112.220492] softirqs last disabled at (291348): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [112.220495] ---[ end trace 0000000000000000 ]---
<6> [112.220496] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.220578] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.220589] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.220744] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.221008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.221148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.221291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.221427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.221562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.221718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.221811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.221906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.221994] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.222093] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.222211] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.223222] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.233573] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.233822] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.234934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.235015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.235091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.235168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.235243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.235319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.235394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.235470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.235545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.235628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.235703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.235779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.235856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.235931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.236007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.236081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.236155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.236229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.236303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.236385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.236465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.236544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.236633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.236712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.236791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.236872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.236953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.237046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.237146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.237240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.237322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.237402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.237489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.237583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.237671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.237753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.237834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.237913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.237991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.238069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.238150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.238233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.238312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.238391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.238473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.238551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.238641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.238718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.238796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.238872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.238949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.239028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.239105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.239182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.239257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.239336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.239417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.239497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.239581] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.239693] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.239697] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24512, lrc_seqno=24512, guc_id=0, flags=0x73 in no process [-1]
<7> [112.239700] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.239759] ------------[ cut here ]------------
<4> [112.239760] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.239761] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.239832] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.239897] autofs4 [last unloaded: snd_hda_intel]
<4> [112.239901] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.239903] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.239905] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.239906] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.239911] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.239981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.239982] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.239984] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.239986] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.239987] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.239988] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.239990] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.239991] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.239992] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.239994] CR2: 00007b19cb31f000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [112.239995] PKRU: 55555554
<4> [112.239996] Call Trace:
<4> [112.239998] <TASK>
<4> [112.240002] ? lock_sync+0x100/0x100
<4> [112.240006] ? lock_release+0xd0/0x2b0
<4> [112.240011] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.240017] process_one_work+0x239/0x760
<4> [112.240023] worker_thread+0x200/0x3f0
<4> [112.240026] ? __pfx_worker_thread+0x10/0x10
<4> [112.240028] kthread+0x10d/0x150
<4> [112.240031] ? __pfx_kthread+0x10/0x10
<4> [112.240035] ret_from_fork+0x3d4/0x480
<4> [112.240037] ? __pfx_kthread+0x10/0x10
<4> [112.240040] ret_from_fork_asm+0x1a/0x30
<4> [112.240047] </TASK>
<4> [112.240048] irq event stamp: 215291
<4> [112.240050] hardirqs last enabled at (215297): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.240052] hardirqs last disabled at (215302): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.240055] softirqs last enabled at (214268): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.240057] softirqs last disabled at (214259): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.240060] ---[ end trace 0000000000000000 ]---
<6> [112.240061] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.240131] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.240137] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.241337] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.241463] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.241582] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.241787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.241879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.241974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.242065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.242153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.242240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.242328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.242420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.242506] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.242611] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.242728] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.243734] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.254456] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.254693] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.255702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.255772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.255842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.255913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.255985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.256057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.256127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.256197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.256265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.256334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.256401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.256469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.256536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.256617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.256691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.256765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.256840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.256916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.256993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.257075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.257156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.257235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.257314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.257392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.257470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.257548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.257639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.257723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.257805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.257884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.257963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.258042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.258129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.258216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.258303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.258386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.258468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.258546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.258637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.258715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.258795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.258876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.258956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.259035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.259116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.259194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.259272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.259348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.259426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.259502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.259580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.259659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.259737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.259814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.259890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.259967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.260042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.260117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.260195] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.260307] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.260311] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24512, lrc_seqno=24512, guc_id=0, flags=0x73 in no process [-1]
<7> [112.260313] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.260372] ------------[ cut here ]------------
<4> [112.260373] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.260374] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.260445] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.260510] autofs4 [last unloaded: snd_hda_intel]
<4> [112.260514] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.260517] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.260518] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.260519] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.260525] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.260604] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.260606] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.260608] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.260610] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.260611] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.260613] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.260614] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.260616] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.260618] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.260619] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.260621] PKRU: 55555554
<4> [112.260622] Call Trace:
<4> [112.260624] <TASK>
<4> [112.260628] ? lock_sync+0x100/0x100
<4> [112.260633] ? lock_release+0xd0/0x2b0
<4> [112.260638] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.260644] process_one_work+0x239/0x760
<4> [112.260651] worker_thread+0x200/0x3f0
<4> [112.260654] ? __pfx_worker_thread+0x10/0x10
<4> [112.260656] kthread+0x10d/0x150
<4> [112.260660] ? __pfx_kthread+0x10/0x10
<4> [112.260663] ret_from_fork+0x3d4/0x480
<4> [112.260666] ? __pfx_kthread+0x10/0x10
<4> [112.260669] ret_from_fork_asm+0x1a/0x30
<4> [112.260677] </TASK>
<4> [112.260678] irq event stamp: 192833
<4> [112.260679] hardirqs last enabled at (192839): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.260682] hardirqs last disabled at (192844): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.260685] softirqs last enabled at (191936): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.260688] softirqs last disabled at (191929): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.260690] ---[ end trace 0000000000000000 ]---
<5> [112.262097] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24567, lrc_seqno=24567, guc_id=0, flags=0x73 in no process [-1]
<7> [112.262101] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.262174] ------------[ cut here ]------------
<4> [112.262175] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.262176] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.262248] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.262311] autofs4 [last unloaded: snd_hda_intel]
<4> [112.262315] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.262317] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.262318] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.262320] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.262325] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.262393] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.262395] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.262397] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.262398] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.262399] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.262401] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.262402] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.262403] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.262404] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.262406] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.262407] PKRU: 55555554
<4> [112.262408] Call Trace:
<4> [112.262409] <TASK>
<4> [112.262413] ? lock_sync+0x100/0x100
<4> [112.262417] ? lock_release+0xd0/0x2b0
<4> [112.262423] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.262428] process_one_work+0x239/0x760
<4> [112.262434] worker_thread+0x200/0x3f0
<4> [112.262437] ? __pfx_worker_thread+0x10/0x10
<4> [112.262439] kthread+0x10d/0x150
<4> [112.262442] ? __pfx_kthread+0x10/0x10
<4> [112.262446] ret_from_fork+0x3d4/0x480
<4> [112.262448] ? __pfx_kthread+0x10/0x10
<4> [112.262451] ret_from_fork_asm+0x1a/0x30
<4> [112.262458] </TASK>
<4> [112.262459] irq event stamp: 194727
<4> [112.262460] hardirqs last enabled at (194733): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.262463] hardirqs last disabled at (194738): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.262465] softirqs last enabled at (191936): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.262467] softirqs last disabled at (191929): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.262470] ---[ end trace 0000000000000000 ]---
<6> [112.262471] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.262539] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.262544] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.262757] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.262860] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.262965] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.263169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.263259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.263352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.263443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.263532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.263628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.263715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.263802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.263884] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.263993] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.264110] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.265118] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.275565] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.275813] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.276818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.276898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.276975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.277052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.277128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.277204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.277280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.277354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.277432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.277509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.277595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.277669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.277744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.277819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.277893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.277967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.278042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.278117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.278192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.278274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.278355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.278437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.278520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.278606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.278685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.278762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.278837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.278916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.278996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.279074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.279152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.279231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.279315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.279400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.279482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.279565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.279644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.279722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.279799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.279875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.279953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.280029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.280106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.280187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.280271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.280348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.280426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.280503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.280587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.280664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.280739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.280818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.280897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.280980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.281058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.281137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.281215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.281292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.281372] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.281483] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.281488] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24567, lrc_seqno=24567, guc_id=0, flags=0x73 in no process [-1]
<7> [112.281490] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.281548] ------------[ cut here ]------------
<4> [112.281549] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.281551] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.281639] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.281704] autofs4 [last unloaded: snd_hda_intel]
<4> [112.281708] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.281711] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.281712] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.281713] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.281718] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.281789] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.281791] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.281793] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.281794] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.281796] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.281797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.281798] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.281799] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.281801] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.281802] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.281804] PKRU: 55555554
<4> [112.281805] Call Trace:
<4> [112.281806] <TASK>
<4> [112.281810] ? lock_sync+0x100/0x100
<4> [112.281814] ? lock_release+0xd0/0x2b0
<4> [112.281819] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.281825] process_one_work+0x239/0x760
<4> [112.281831] worker_thread+0x200/0x3f0
<4> [112.281834] ? __pfx_worker_thread+0x10/0x10
<4> [112.281836] kthread+0x10d/0x150
<4> [112.281839] ? __pfx_kthread+0x10/0x10
<4> [112.281843] ret_from_fork+0x3d4/0x480
<4> [112.281844] ? __pfx_kthread+0x10/0x10
<4> [112.281848] ret_from_fork_asm+0x1a/0x30
<4> [112.281855] </TASK>
<4> [112.281856] irq event stamp: 197831
<4> [112.281857] hardirqs last enabled at (197837): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.281860] hardirqs last disabled at (197842): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.281862] softirqs last enabled at (197088): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.281865] softirqs last disabled at (197075): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.281867] ---[ end trace 0000000000000000 ]---
<6> [112.282796] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.282872] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.282878] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.282931] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.283035] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.283139] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.283340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.283429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.283523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.283619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.283705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.283790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.283875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.283962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.284047] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.284143] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.284259] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.285266] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.295559] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.295807] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.296818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.296898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.296977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.297056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.297133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.297209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.297285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.297360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.297436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.297512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.297597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.297671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.297747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.297821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.297896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.297970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.298043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.298119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.298194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.298277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.298359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.298438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.298518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.298604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.298681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.298757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.298833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.298913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.298993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.299075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.299159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.299240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.299325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.299409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.299494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.299578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.299658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.299736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.299814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.299890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.299969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.300046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.300122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.300198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.300278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.300354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.300431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.300507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.300594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.300669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.300746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.300829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.300910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.300990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.301067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.301147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.301223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.301300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.301379] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.301490] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.301494] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24567, lrc_seqno=24567, guc_id=0, flags=0x73 in no process [-1]
<7> [112.301497] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.301559] ------------[ cut here ]------------
<4> [112.301561] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.301562] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.301634] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.301698] autofs4 [last unloaded: snd_hda_intel]
<4> [112.301701] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.301704] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.301705] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.301706] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.301712] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.301782] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.301784] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.301786] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.301787] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.301789] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.301790] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.301791] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.301792] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.301794] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.301795] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.301797] PKRU: 55555554
<4> [112.301798] Call Trace:
<4> [112.301799] <TASK>
<4> [112.301803] ? lock_sync+0x100/0x100
<4> [112.301807] ? lock_release+0xd0/0x2b0
<4> [112.301812] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.301818] process_one_work+0x239/0x760
<4> [112.301824] worker_thread+0x200/0x3f0
<4> [112.301827] ? __pfx_worker_thread+0x10/0x10
<4> [112.301829] kthread+0x10d/0x150
<4> [112.301832] ? __pfx_kthread+0x10/0x10
<4> [112.301836] ret_from_fork+0x3d4/0x480
<4> [112.301838] ? __pfx_kthread+0x10/0x10
<4> [112.301841] ret_from_fork_asm+0x1a/0x30
<4> [112.301848] </TASK>
<4> [112.301849] irq event stamp: 200935
<4> [112.301850] hardirqs last enabled at (200941): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.301853] hardirqs last disabled at (200946): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.301855] softirqs last enabled at (200026): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.301858] softirqs last disabled at (200019): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.301860] ---[ end trace 0000000000000000 ]---
<5> [112.303094] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24622, lrc_seqno=24622, guc_id=0, flags=0x73 in no process [-1]
<7> [112.303097] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.303160] ------------[ cut here ]------------
<4> [112.303161] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.303163] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.303233] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.303296] autofs4 [last unloaded: snd_hda_intel]
<4> [112.303300] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.303302] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.303303] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.303305] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.303309] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.303379] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.303380] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.303383] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.303384] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.303386] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.303387] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.303389] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.303390] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.303392] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.303393] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.303395] PKRU: 55555554
<4> [112.303396] Call Trace:
<4> [112.303397] <TASK>
<4> [112.303401] ? lock_sync+0x100/0x100
<4> [112.303406] ? lock_release+0xd0/0x2b0
<4> [112.303411] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.303417] process_one_work+0x239/0x760
<4> [112.303423] worker_thread+0x200/0x3f0
<4> [112.303426] ? __pfx_worker_thread+0x10/0x10
<4> [112.303428] kthread+0x10d/0x150
<4> [112.303432] ? __pfx_kthread+0x10/0x10
<4> [112.303435] ret_from_fork+0x3d4/0x480
<4> [112.303437] ? __pfx_kthread+0x10/0x10
<4> [112.303441] ret_from_fork_asm+0x1a/0x30
<4> [112.303449] </TASK>
<4> [112.303450] irq event stamp: 202793
<4> [112.303451] hardirqs last enabled at (202799): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.303454] hardirqs last disabled at (202804): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.303456] softirqs last enabled at (200026): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.303459] softirqs last disabled at (200019): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.303461] ---[ end trace 0000000000000000 ]---
<6> [112.303463] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.303531] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.303537] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.303770] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.303874] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.303981] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.304183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.304274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.304368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.304457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.304545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.304647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.304733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.304820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.304902] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.304997] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.305118] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.306125] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.316557] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.316806] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.317825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.317904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.317979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.318057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.318133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.318208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.318283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.318360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.318435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.318510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.318594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.318669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.318744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.318818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.318892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.318966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.319039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.319113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.319188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.319271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.319353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.319433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.319516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.319608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.319690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.319768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.319847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.319933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.320018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.320099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.320180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.320260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.320346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.320434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.320523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.320616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.320698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.320778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.320857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.320933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.321014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.321095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.321178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.321258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.321341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.321419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.321497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.321580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.321657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.321733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.321809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.321887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.321965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.322043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.322120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.322198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.322277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.322359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.322440] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.323394] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.323398] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24622, lrc_seqno=24622, guc_id=0, flags=0x73 in no process [-1]
<7> [112.323401] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.323465] ------------[ cut here ]------------
<4> [112.323466] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.323467] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.323539] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.323608] autofs4 [last unloaded: snd_hda_intel]
<4> [112.323612] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.323614] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.323616] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.323617] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.323622] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.323691] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.323692] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.323695] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.323696] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.323697] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.323698] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.323700] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.323701] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.323703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.323704] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.323705] PKRU: 55555554
<4> [112.323707] Call Trace:
<4> [112.323708] <TASK>
<4> [112.323711] ? lock_sync+0x100/0x100
<4> [112.323716] ? lock_release+0xd0/0x2b0
<4> [112.323721] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.323727] process_one_work+0x239/0x760
<4> [112.323733] worker_thread+0x200/0x3f0
<4> [112.323735] ? __pfx_worker_thread+0x10/0x10
<4> [112.323738] kthread+0x10d/0x150
<4> [112.323741] ? __pfx_kthread+0x10/0x10
<4> [112.323744] ret_from_fork+0x3d4/0x480
<4> [112.323746] ? __pfx_kthread+0x10/0x10
<4> [112.323750] ret_from_fork_asm+0x1a/0x30
<4> [112.323757] </TASK>
<4> [112.323758] irq event stamp: 205903
<4> [112.323759] hardirqs last enabled at (205909): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.323762] hardirqs last disabled at (205914): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.323764] softirqs last enabled at (204896): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.323767] softirqs last disabled at (204889): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.323769] ---[ end trace 0000000000000000 ]---
<6> [112.323771] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.323840] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.323846] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.323860] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.323962] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.324257] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.324468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.324566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.324664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.324755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.324842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.324929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.325014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.325100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.325182] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.325274] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.325389] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.326415] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.336551] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.336800] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.337762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.337842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.337920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.337999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.338077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.338155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.338235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.338314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.338390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.338468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.338551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.338628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.338703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.338779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.338854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.338928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.339002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.339077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.339150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.339233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.339316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.339399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.339484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.339571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.339652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.339730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.339810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.339890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.339971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.340053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.340136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.340217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.340302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.340388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.340472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.340556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.340637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.340717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.340800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.340882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.340965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.341048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.341130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.341211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.341295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.341374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.341451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.341528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.341620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.341698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.341774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.341856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.341939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.342021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.342101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.342182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.342262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.342340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.342420] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.342533] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.342538] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24622, lrc_seqno=24622, guc_id=0, flags=0x73 in no process [-1]
<7> [112.342540] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.342622] ------------[ cut here ]------------
<4> [112.342624] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.342626] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:157/6437
<4> [112.342699] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.342765] autofs4 [last unloaded: snd_hda_intel]
<4> [112.342770] CPU: 4 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.342773] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.342774] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.342775] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.342781] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.342850] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.342852] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.342854] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.342856] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.342857] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.342858] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.342859] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.342861] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [112.342862] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.342864] CR2: 000074f6a6b1f008 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.342865] PKRU: 55555554
<4> [112.342866] Call Trace:
<4> [112.342868] <TASK>
<4> [112.342872] ? lock_sync+0x100/0x100
<4> [112.342876] ? lock_release+0xd0/0x2b0
<4> [112.342882] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.342887] process_one_work+0x239/0x760
<4> [112.342894] worker_thread+0x200/0x3f0
<4> [112.342897] ? __pfx_worker_thread+0x10/0x10
<4> [112.342899] kthread+0x10d/0x150
<4> [112.342902] ? __pfx_kthread+0x10/0x10
<4> [112.342906] ret_from_fork+0x3d4/0x480
<4> [112.342908] ? __pfx_kthread+0x10/0x10
<4> [112.342911] ret_from_fork_asm+0x1a/0x30
<4> [112.342919] </TASK>
<4> [112.342920] irq event stamp: 301555
<4> [112.342921] hardirqs last enabled at (301561): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.342924] hardirqs last disabled at (301566): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.342926] softirqs last enabled at (300794): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.342929] softirqs last disabled at (300787): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.342931] ---[ end trace 0000000000000000 ]---
<5> [112.344450] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24677, lrc_seqno=24677, guc_id=0, flags=0x73 in no process [-1]
<7> [112.344455] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [112.344572] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.344723] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.344869] ------------[ cut here ]------------
<4> [112.344871] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.344873] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:157/6437
<4> [112.344983] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.345111] autofs4 [last unloaded: snd_hda_intel]
<4> [112.345118] CPU: 4 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.345123] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.345125] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.345127] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.345135] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.345242] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.345246] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.345250] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.345252] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.345255] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.345258] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.345260] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.345263] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [112.345266] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.345268] CR2: 000074f6a6b1f008 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.345271] PKRU: 55555554
<4> [112.345273] Call Trace:
<4> [112.345275] <TASK>
<4> [112.345282] ? lock_sync+0x100/0x100
<4> [112.345290] ? lock_release+0xd0/0x2b0
<4> [112.345300] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.345311] process_one_work+0x239/0x760
<4> [112.345322] worker_thread+0x200/0x3f0
<4> [112.345327] ? __pfx_worker_thread+0x10/0x10
<4> [112.345332] kthread+0x10d/0x150
<4> [112.345337] ? __pfx_kthread+0x10/0x10
<4> [112.345343] ret_from_fork+0x3d4/0x480
<4> [112.345347] ? __pfx_kthread+0x10/0x10
<4> [112.345352] ret_from_fork_asm+0x1a/0x30
<4> [112.345366] </TASK>
<4> [112.345368] irq event stamp: 303083
<4> [112.345371] hardirqs last enabled at (303089): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.345375] hardirqs last disabled at (303094): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.345378] softirqs last enabled at (300794): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.345383] softirqs last disabled at (300787): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.345386] ---[ end trace 0000000000000000 ]---
<6> [112.345389] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.345495] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.345505] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.345748] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.345996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.346129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.346266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.346399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.346529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.346681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.346810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.346940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.347064] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.347200] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.347362] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.348474] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.358556] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.358816] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.359867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.359949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.360028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.360107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.360185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.360262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.360342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.360421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.360498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.360586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.360662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.360737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.360812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.360886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.360960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.361034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.361108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.361181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.361255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.361338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.361418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.361498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.361605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.361687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.361768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.361846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.361923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.362004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.362086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.362165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.362245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.362323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.362407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.362492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.362591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.362677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.362759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.362838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.362916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.362992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.363073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.363150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.363229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.363309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.363392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.363471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.363554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.363632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.363711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.363788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.363868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.363951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.364030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.364109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.364184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.364263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.364338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.364414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.364493] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.364615] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.364620] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24677, lrc_seqno=24677, guc_id=0, flags=0x73 in no process [-1]
<7> [112.364622] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.364681] ------------[ cut here ]------------
<4> [112.364683] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.364684] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:157/6437
<4> [112.364756] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.364824] autofs4 [last unloaded: snd_hda_intel]
<4> [112.364829] CPU: 4 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.364832] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.364833] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.364835] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.364840] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.364910] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.364912] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.364914] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.364916] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.364918] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.364919] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.364921] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.364922] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [112.364924] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.364926] CR2: 000074f6a6b1f008 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.364927] PKRU: 55555554
<4> [112.364929] Call Trace:
<4> [112.364930] <TASK>
<4> [112.364934] ? lock_sync+0x100/0x100
<4> [112.364939] ? lock_release+0xd0/0x2b0
<4> [112.364945] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.364951] process_one_work+0x239/0x760
<4> [112.364957] worker_thread+0x200/0x3f0
<4> [112.364960] ? __pfx_worker_thread+0x10/0x10
<4> [112.364963] kthread+0x10d/0x150
<4> [112.364966] ? __pfx_kthread+0x10/0x10
<4> [112.364969] ret_from_fork+0x3d4/0x480
<4> [112.364971] ? __pfx_kthread+0x10/0x10
<4> [112.364975] ret_from_fork_asm+0x1a/0x30
<4> [112.364982] </TASK>
<4> [112.364983] irq event stamp: 306191
<4> [112.364984] hardirqs last enabled at (306197): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.364987] hardirqs last disabled at (306202): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.364989] softirqs last enabled at (305178): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.364992] softirqs last disabled at (305171): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.364994] ---[ end trace 0000000000000000 ]---
<6> [112.364996] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.365065] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.365071] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.366321] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.366427] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.366534] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.366756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.366846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.366938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.367027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.367113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.367198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.367284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.367371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.367454] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.367552] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.367671] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.368678] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.379400] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.379643] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.380678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.380758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.380835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.380913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.380990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.381067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.381141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.381218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.381293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.381372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.381450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.381526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.381617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.381693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.381768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.381842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.381917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.381991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.382064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.382148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.382231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.382310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.382390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.382468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.382548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.382625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.382701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.382781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.382865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.382945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.383025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.383105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.383195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.383282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.383369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.383450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.383530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.383621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.383697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.383773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.383856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.383939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.384020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.384099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.384181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.384259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.384337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.384413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.384491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.384576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.384658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.384740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.384819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.384897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.384974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.385052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.385127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.385203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.385282] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.385393] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.385398] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24677, lrc_seqno=24677, guc_id=0, flags=0x73 in no process [-1]
<7> [112.385400] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.385459] ------------[ cut here ]------------
<4> [112.385460] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.385462] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.385538] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.385604] autofs4 [last unloaded: snd_hda_intel]
<4> [112.385608] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.385611] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.385612] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.385613] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.385618] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.385689] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.385690] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.385692] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.385694] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.385695] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.385696] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.385697] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.385699] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.385700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.385702] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.385703] PKRU: 55555554
<4> [112.385704] Call Trace:
<4> [112.385705] <TASK>
<4> [112.385709] ? lock_sync+0x100/0x100
<4> [112.385714] ? lock_release+0xd0/0x2b0
<4> [112.385719] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.385725] process_one_work+0x239/0x760
<4> [112.385731] worker_thread+0x200/0x3f0
<4> [112.385734] ? __pfx_worker_thread+0x10/0x10
<4> [112.385736] kthread+0x10d/0x150
<4> [112.385739] ? __pfx_kthread+0x10/0x10
<4> [112.385742] ret_from_fork+0x3d4/0x480
<4> [112.385744] ? __pfx_kthread+0x10/0x10
<4> [112.385748] ret_from_fork_asm+0x1a/0x30
<4> [112.385755] </TASK>
<4> [112.385756] irq event stamp: 209557
<4> [112.385757] hardirqs last enabled at (209563): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.385760] hardirqs last disabled at (209568): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.385762] softirqs last enabled at (208570): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.385765] softirqs last disabled at (208563): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.385767] ---[ end trace 0000000000000000 ]---
<5> [112.386971] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24732, lrc_seqno=24732, guc_id=0, flags=0x73 in no process [-1]
<7> [112.386974] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.387035] ------------[ cut here ]------------
<4> [112.387036] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.387038] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.387107] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.387182] autofs4 [last unloaded: snd_hda_intel]
<4> [112.387187] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.387189] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.387190] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.387192] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.387197] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.387267] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.387268] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.387270] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.387272] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.387273] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.387274] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.387275] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.387277] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.387278] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.387279] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.387281] PKRU: 55555554
<4> [112.387282] Call Trace:
<4> [112.387283] <TASK>
<4> [112.387287] ? lock_sync+0x100/0x100
<4> [112.387291] ? lock_release+0xd0/0x2b0
<4> [112.387296] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.387302] process_one_work+0x239/0x760
<4> [112.387307] worker_thread+0x200/0x3f0
<4> [112.387310] ? __pfx_worker_thread+0x10/0x10
<4> [112.387312] kthread+0x10d/0x150
<4> [112.387315] ? __pfx_kthread+0x10/0x10
<4> [112.387319] ret_from_fork+0x3d4/0x480
<4> [112.387321] ? __pfx_kthread+0x10/0x10
<4> [112.387324] ret_from_fork_asm+0x1a/0x30
<4> [112.387331] </TASK>
<4> [112.387332] irq event stamp: 211415
<4> [112.387333] hardirqs last enabled at (211421): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.387336] hardirqs last disabled at (211426): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.387338] softirqs last enabled at (208570): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.387341] softirqs last disabled at (208563): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.387343] ---[ end trace 0000000000000000 ]---
<6> [112.387345] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.387413] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.387420] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.387520] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.387647] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.387756] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.388010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.388142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.388280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.388412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.388556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.388648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.388739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.388832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.388919] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.389015] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.389133] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.390154] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.400544] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.400792] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.401792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.401872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.401950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.402027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.402104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.402179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.402255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.402329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.402406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.402482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.402563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.402639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.402713] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.402788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.402865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.402942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.403018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.403092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.403166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.403247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.403328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.403406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.403486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.403572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.403650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.403730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.403811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.403894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.403975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.404054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.404133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.404213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.404297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.404381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.404466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.404554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.404638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.404717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.404795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.404872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.404951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.405029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.405111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.405189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.405269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.405347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.405425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.405502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.405588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.405669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.405748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.405828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.405907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.405983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.406060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.406138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.406215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.406294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.406375] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.406488] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.406492] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24732, lrc_seqno=24732, guc_id=0, flags=0x73 in no process [-1]
<7> [112.406494] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.407404] ------------[ cut here ]------------
<4> [112.407405] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.407407] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:157/6437
<4> [112.407482] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.407553] autofs4 [last unloaded: snd_hda_intel]
<4> [112.407557] CPU: 6 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.407560] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.407561] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.407562] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.407568] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.407638] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.407640] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.407642] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.407644] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.407645] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.407647] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.407648] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.407650] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.407652] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.407653] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.407655] PKRU: 55555554
<4> [112.407656] Call Trace:
<4> [112.407658] <TASK>
<4> [112.407662] ? lock_sync+0x100/0x100
<4> [112.407666] ? lock_release+0xd0/0x2b0
<4> [112.407672] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.407678] process_one_work+0x239/0x760
<4> [112.407685] worker_thread+0x200/0x3f0
<4> [112.407688] ? __pfx_worker_thread+0x10/0x10
<4> [112.407690] kthread+0x10d/0x150
<4> [112.407693] ? __pfx_kthread+0x10/0x10
<4> [112.407697] ret_from_fork+0x3d4/0x480
<4> [112.407699] ? __pfx_kthread+0x10/0x10
<4> [112.407703] ret_from_fork_asm+0x1a/0x30
<4> [112.407711] </TASK>
<4> [112.407712] irq event stamp: 310005
<4> [112.407713] hardirqs last enabled at (310011): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.407716] hardirqs last disabled at (310016): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.407719] softirqs last enabled at (308996): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.407722] softirqs last disabled at (308987): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.407724] ---[ end trace 0000000000000000 ]---
<6> [112.407726] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.407796] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.407802] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.407816] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.407920] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.408254] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.408464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.408561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.408656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.408744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.408830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.408915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.409001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.409089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.409172] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.409267] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.409387] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.410394] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.420562] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.420814] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.421858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.421949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.422048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.422138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.422216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.422292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.422368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.422442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.422518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.422609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.422684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.422758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.422834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.422910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.422988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.423066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.423142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.423218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.423292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.423375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.423457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.423544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.423625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.423705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.423782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.423861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.423940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.424025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.424111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.424193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.424275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.424356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.424443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.424533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.424617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.424698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.424797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.424890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.424978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.425059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.425143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.425224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.425303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.425380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.425462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.425550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.425632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.425713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.425793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.425872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.425949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.426033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.426113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.426193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.426272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.426350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.426427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.426503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.426593] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.426713] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.426718] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24732, lrc_seqno=24732, guc_id=0, flags=0x73 in no process [-1]
<7> [112.426721] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.426780] ------------[ cut here ]------------
<4> [112.426781] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.426782] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:155/6435
<4> [112.426855] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.426941] autofs4 [last unloaded: snd_hda_intel]
<4> [112.426946] CPU: 10 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.426949] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.426950] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.426951] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.426957] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.427026] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.427028] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.427030] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.427031] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.427033] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.427034] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.427035] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.427036] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [112.427038] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.427039] CR2: 00007b19cb317000 CR3: 000000015b49d004 CR4: 0000000000f72ef0
<4> [112.427041] PKRU: 55555554
<4> [112.427042] Call Trace:
<4> [112.427043] <TASK>
<4> [112.427047] ? lock_sync+0x100/0x100
<4> [112.427052] ? lock_release+0xd0/0x2b0
<4> [112.427057] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.427063] process_one_work+0x239/0x760
<4> [112.427070] worker_thread+0x200/0x3f0
<4> [112.427072] ? __pfx_worker_thread+0x10/0x10
<4> [112.427075] kthread+0x10d/0x150
<4> [112.427078] ? __pfx_kthread+0x10/0x10
<4> [112.427081] ret_from_fork+0x3d4/0x480
<4> [112.427083] ? __pfx_kthread+0x10/0x10
<4> [112.427087] ret_from_fork_asm+0x1a/0x30
<4> [112.427094] </TASK>
<4> [112.427095] irq event stamp: 214931
<4> [112.427096] hardirqs last enabled at (214937): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.427099] hardirqs last disabled at (214942): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.427101] softirqs last enabled at (214088): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.427104] softirqs last disabled at (214083): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.427106] ---[ end trace 0000000000000000 ]---
<5> [112.428348] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24787, lrc_seqno=24787, guc_id=0, flags=0x73 in no process [-1]
<7> [112.428351] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.428411] ------------[ cut here ]------------
<4> [112.428412] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.428414] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:155/6435
<4> [112.428487] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.428561] autofs4 [last unloaded: snd_hda_intel]
<4> [112.428565] CPU: 10 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.428568] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.428570] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.428571] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.428576] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.428650] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.428652] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.428654] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.428656] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.428657] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.428659] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.428660] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.428662] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [112.428663] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.428665] CR2: 00007b19cb317000 CR3: 000000015b49d004 CR4: 0000000000f72ef0
<4> [112.428667] PKRU: 55555554
<4> [112.428668] Call Trace:
<4> [112.428669] <TASK>
<4> [112.428673] ? lock_sync+0x100/0x100
<4> [112.428677] ? lock_release+0xd0/0x2b0
<4> [112.428683] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.428689] process_one_work+0x239/0x760
<4> [112.428695] worker_thread+0x200/0x3f0
<4> [112.428698] ? __pfx_worker_thread+0x10/0x10
<4> [112.428700] kthread+0x10d/0x150
<4> [112.428703] ? __pfx_kthread+0x10/0x10
<4> [112.428707] ret_from_fork+0x3d4/0x480
<4> [112.428709] ? __pfx_kthread+0x10/0x10
<4> [112.428713] ret_from_fork_asm+0x1a/0x30
<4> [112.428720] </TASK>
<4> [112.428722] irq event stamp: 216801
<4> [112.428723] hardirqs last enabled at (216807): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.428726] hardirqs last disabled at (216812): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.428728] softirqs last enabled at (216366): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.428731] softirqs last disabled at (216359): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.428733] ---[ end trace 0000000000000000 ]---
<7> [112.428749] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.428854] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [112.428953] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.429024] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.429032] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.429085] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.429348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.429440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.429554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.429647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.429736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.429824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.429909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.429997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.430080] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.430175] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.430294] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.431305] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.441533] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.441784] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.442816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.442903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.442982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.443061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.443138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.443214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.443290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.443369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.443447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.443529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.443605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.443681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.443755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.443831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.443906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.443982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.444061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.444138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.444214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.444299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.444381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.444463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.444552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.444633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.444712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.444791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.444868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.444947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.445028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.445106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.445184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.445267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.445358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.445446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.445537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.445619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.445700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.445779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.445859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.445941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.446024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.446103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.446183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.446261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.446342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.446419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.446498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.446582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.446660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.446740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.446820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.446903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.446982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.447060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.447137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.447215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.447291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.447368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.447447] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.447562] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.447567] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24787, lrc_seqno=24787, guc_id=0, flags=0x73 in no process [-1]
<7> [112.447569] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.447629] ------------[ cut here ]------------
<4> [112.447630] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.447632] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.447706] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.447777] autofs4 [last unloaded: snd_hda_intel]
<4> [112.447781] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.447784] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.447785] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.447787] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.447792] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.447863] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.447865] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.447867] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.447868] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.447870] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.447871] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.447872] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.447873] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.447875] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.447876] CR2: 00007b19cb31f000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [112.447878] PKRU: 55555554
<4> [112.447879] Call Trace:
<4> [112.447880] <TASK>
<4> [112.447884] ? lock_sync+0x100/0x100
<4> [112.447888] ? lock_release+0xd0/0x2b0
<4> [112.447894] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.447899] process_one_work+0x239/0x760
<4> [112.447906] worker_thread+0x200/0x3f0
<4> [112.447908] ? __pfx_worker_thread+0x10/0x10
<4> [112.447911] kthread+0x10d/0x150
<4> [112.447914] ? __pfx_kthread+0x10/0x10
<4> [112.447917] ret_from_fork+0x3d4/0x480
<4> [112.447919] ? __pfx_kthread+0x10/0x10
<4> [112.447923] ret_from_fork_asm+0x1a/0x30
<4> [112.447930] </TASK>
<4> [112.447931] irq event stamp: 223731
<4> [112.447932] hardirqs last enabled at (223737): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.447935] hardirqs last disabled at (223742): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.447937] softirqs last enabled at (222718): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.447940] softirqs last disabled at (222711): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.447942] ---[ end trace 0000000000000000 ]---
<6> [112.447944] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.448013] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.448019] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.448559] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.448762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.448853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.448948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.449035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.449125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.449212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.449300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.449393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.449480] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.449587] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.449705] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.450712] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.461435] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.461685] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.462702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.462781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.462858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.462938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.463016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.463094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.463171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.463247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.463327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.463405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.463483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.463571] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.463647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.463723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.463798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.463873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.463948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.464023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.464098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.464181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.464265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.464346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.464428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.464507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.464598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.464675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.464753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.464833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.464914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.464996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.465079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.465161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.465248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.465334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.465421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.465503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.465596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.465675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.465754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.465831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.465912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.465994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.466077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.466159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.466243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.466322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.466404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.466484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.466574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.466652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.466729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.466810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.466888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.466970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.467050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.467130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.467207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.467285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.467366] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.467476] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.467480] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24787, lrc_seqno=24787, guc_id=0, flags=0x73 in no process [-1]
<7> [112.467483] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.468395] ------------[ cut here ]------------
<4> [112.468397] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.468398] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.468471] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.468542] autofs4 [last unloaded: snd_hda_intel]
<4> [112.468546] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.468549] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.468550] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.468552] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.468557] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.468627] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.468628] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.468630] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.468632] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.468633] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.468634] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.468635] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.468637] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.468638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.468640] CR2: 00007b19cb31f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.468641] PKRU: 55555554
<4> [112.468642] Call Trace:
<4> [112.468643] <TASK>
<4> [112.468647] ? lock_sync+0x100/0x100
<4> [112.468652] ? lock_release+0xd0/0x2b0
<4> [112.468657] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.468663] process_one_work+0x239/0x760
<4> [112.468669] worker_thread+0x200/0x3f0
<4> [112.468672] ? __pfx_worker_thread+0x10/0x10
<4> [112.468674] kthread+0x10d/0x150
<4> [112.468677] ? __pfx_kthread+0x10/0x10
<4> [112.468681] ret_from_fork+0x3d4/0x480
<4> [112.468683] ? __pfx_kthread+0x10/0x10
<4> [112.468686] ret_from_fork_asm+0x1a/0x30
<4> [112.468693] </TASK>
<4> [112.468694] irq event stamp: 226851
<4> [112.468695] hardirqs last enabled at (226857): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.468698] hardirqs last disabled at (226862): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.468700] softirqs last enabled at (226482): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.468703] softirqs last disabled at (226471): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.468705] ---[ end trace 0000000000000000 ]---
<7> [112.468719] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.468823] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [112.469266] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24842, lrc_seqno=24842, guc_id=0, flags=0x73 in no process [-1]
<7> [112.469269] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.469329] ------------[ cut here ]------------
<4> [112.469330] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.469331] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:580/6860
<4> [112.469404] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.469468] autofs4 [last unloaded: snd_hda_intel]
<4> [112.469471] CPU: 8 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.469474] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.469475] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.469476] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.469481] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.469562] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.469564] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.469566] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.469567] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.469569] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.469570] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.469572] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.469573] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.469575] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.469576] CR2: 00007b19cb31f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.469578] PKRU: 55555554
<4> [112.469579] Call Trace:
<4> [112.469580] <TASK>
<4> [112.469584] ? lock_sync+0x100/0x100
<4> [112.469588] ? lock_release+0xd0/0x2b0
<4> [112.469594] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.469599] process_one_work+0x239/0x760
<4> [112.469606] worker_thread+0x200/0x3f0
<4> [112.469608] ? __pfx_worker_thread+0x10/0x10
<4> [112.469611] kthread+0x10d/0x150
<4> [112.469614] ? __pfx_kthread+0x10/0x10
<4> [112.469618] ret_from_fork+0x3d4/0x480
<4> [112.469620] ? __pfx_kthread+0x10/0x10
<4> [112.469623] ret_from_fork_asm+0x1a/0x30
<4> [112.469630] </TASK>
<4> [112.469632] irq event stamp: 228729
<4> [112.469633] hardirqs last enabled at (228735): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.469635] hardirqs last disabled at (228740): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.469637] softirqs last enabled at (228568): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.469640] softirqs last disabled at (228559): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.469642] ---[ end trace 0000000000000000 ]---
<6> [112.469644] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.469714] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.469720] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.469919] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.470125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.470216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.470311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.470400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.470491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.470591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.470679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.470767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.470849] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.470944] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.471061] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.472083] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.482524] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.482773] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.483786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.483866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.483943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.484020] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.484097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.484174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.484250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.484326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.484402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.484477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.484568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.484648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.484725] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.484802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.484878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.484954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.485027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.485102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.485177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.485258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.485341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.485421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.485502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.485598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.485678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.485758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.485837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.485917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.485997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.486077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.486156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.486235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.486319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.486401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.486490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.486584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.486666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.486745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.486823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.486900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.486979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.487057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.487138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.487217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.487299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.487377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.487456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.487538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.487616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.487693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.487769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.487849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.487931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.488012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.488090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.488168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.488244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.488320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.488399] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.489522] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.489529] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24842, lrc_seqno=24842, guc_id=0, flags=0x73 in no process [-1]
<7> [112.489532] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.489599] ------------[ cut here ]------------
<4> [112.489600] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.489601] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.489678] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.489744] autofs4 [last unloaded: snd_hda_intel]
<4> [112.489748] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.489751] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<7> [112.489702] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [112.489753] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.489754] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.489759] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.489831] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.489833] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.489836] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.489838] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.489839] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.489840] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.489842] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.489843] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.489845] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.489847] CR2: 00007b19cb31b000 CR3: 000000015b49d003 CR4: 0000000000f72ef0
<4> [112.489849] PKRU: 55555554
<4> [112.489850] Call Trace:
<4> [112.489851] <TASK>
<7> [112.489807] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.489855] ? lock_sync+0x100/0x100
<4> [112.489860] ? lock_release+0xd0/0x2b0
<4> [112.489866] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.489871] process_one_work+0x239/0x760
<4> [112.489878] worker_thread+0x200/0x3f0
<4> [112.489880] ? __pfx_worker_thread+0x10/0x10
<4> [112.489883] kthread+0x10d/0x150
<4> [112.489886] ? __pfx_kthread+0x10/0x10
<4> [112.489889] ret_from_fork+0x3d4/0x480
<4> [112.489891] ? __pfx_kthread+0x10/0x10
<4> [112.489895] ret_from_fork_asm+0x1a/0x30
<4> [112.489902] </TASK>
<4> [112.489903] irq event stamp: 220253
<4> [112.489911] hardirqs last enabled at (220259): [<ffffffff814b0064>] vprintk_emit+0x414/0x470
<4> [112.489914] hardirqs last disabled at (220264): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.489916] softirqs last enabled at (219456): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.489919] softirqs last disabled at (219445): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.489922] ---[ end trace 0000000000000000 ]---
<6> [112.489924] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.489993] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.490000] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.490409] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.490617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.490706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.490799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.490886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.490973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.491060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.491147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.491238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.491326] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.491423] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.491544] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.492559] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.502521] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.502770] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.503760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.503841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.503920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.503999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.504079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.504156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.504233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.504309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.504384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.504459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.504540] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.504614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.504688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.504763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.504836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.504909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.504982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.505057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.505130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.505213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.505295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.505375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.505454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.505539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.505620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.505700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.505778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.505860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.505939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.506018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.506097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.506177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.506263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.506351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.506439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.506529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.506616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.506697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.506776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.506854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.506938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.507020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.507101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.507179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.507261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.507339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.507417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.507494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.507583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.507660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.507736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.507815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.507893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.507971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.508051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.508133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.508212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.508290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.508371] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.508486] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.508490] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24842, lrc_seqno=24842, guc_id=0, flags=0x73 in no process [-1]
<7> [112.508492] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.508573] ------------[ cut here ]------------
<4> [112.508575] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.508576] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:157/6437
<4> [112.508650] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.508727] autofs4 [last unloaded: snd_hda_intel]
<4> [112.508732] CPU: 11 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.508735] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.508737] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.508738] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.508744] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.508815] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.508816] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.508819] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.508821] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.508822] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.508824] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.508825] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.508827] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [112.508828] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.508830] CR2: 00005c658006a0b0 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.508832] PKRU: 55555554
<4> [112.508833] Call Trace:
<4> [112.508834] <TASK>
<4> [112.508839] ? lock_sync+0x100/0x100
<4> [112.508844] ? lock_release+0xd0/0x2b0
<4> [112.508849] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.508855] process_one_work+0x239/0x760
<4> [112.508862] worker_thread+0x200/0x3f0
<4> [112.508865] ? __pfx_worker_thread+0x10/0x10
<4> [112.508867] kthread+0x10d/0x150
<4> [112.508870] ? __pfx_kthread+0x10/0x10
<4> [112.508874] ret_from_fork+0x3d4/0x480
<4> [112.508876] ? __pfx_kthread+0x10/0x10
<4> [112.508879] ret_from_fork_asm+0x1a/0x30
<4> [112.508886] </TASK>
<4> [112.508888] irq event stamp: 317115
<4> [112.508889] hardirqs last enabled at (317121): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.508892] hardirqs last disabled at (317126): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.508894] softirqs last enabled at (316354): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.508896] softirqs last disabled at (316337): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.508899] ---[ end trace 0000000000000000 ]---
<5> [112.510302] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24897, lrc_seqno=24897, guc_id=0, flags=0x73 in no process [-1]
<7> [112.510306] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.510377] ------------[ cut here ]------------
<4> [112.510378] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.510380] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:157/6437
<4> [112.510449] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.510517] autofs4 [last unloaded: snd_hda_intel]
<4> [112.510521] CPU: 11 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.510524] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.510525] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.510526] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.510531] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.510600] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.510601] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.510603] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.510604] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.510606] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.510607] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.510608] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.510609] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [112.510611] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.510612] CR2: 00005c658006a0b0 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.510614] PKRU: 55555554
<4> [112.510615] Call Trace:
<4> [112.510616] <TASK>
<4> [112.510619] ? lock_sync+0x100/0x100
<4> [112.510623] ? lock_release+0xd0/0x2b0
<4> [112.510629] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.510634] process_one_work+0x239/0x760
<4> [112.510640] worker_thread+0x200/0x3f0
<4> [112.510643] ? __pfx_worker_thread+0x10/0x10
<4> [112.510645] kthread+0x10d/0x150
<4> [112.510648] ? __pfx_kthread+0x10/0x10
<4> [112.510652] ret_from_fork+0x3d4/0x480
<4> [112.510655] ? __pfx_kthread+0x10/0x10
<4> [112.510658] ret_from_fork_asm+0x1a/0x30
<4> [112.510665] </TASK>
<4> [112.510667] irq event stamp: 319407
<4> [112.510668] hardirqs last enabled at (319413): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.510671] hardirqs last disabled at (319418): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.510673] softirqs last enabled at (317514): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [112.510676] softirqs last disabled at (317512): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [112.510679] ---[ end trace 0000000000000000 ]---
<7> [112.510693] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.510797] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [112.510896] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.510967] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.510976] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.511023] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.511236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.511327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.511422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.511515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.511604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.511690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.511777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.511865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.511948] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.512047] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.512165] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.513188] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.523516] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.523765] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.524791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.524872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.524949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.525026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.525102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.525177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.525253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.525329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.525404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.525480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.525564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.525641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.525717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.525795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.525871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.525947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.526022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.526096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.526171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.526252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.526334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.526415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.526498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.526590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.526669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.526746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.526823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.526903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.526982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.527061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.527144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.527225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.527309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.527393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.527477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.527566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.527646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.527723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.527800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.527878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.527961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.528042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.528121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.528199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.528280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.528358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.528438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.528520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.528601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.528679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.528754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.528833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.528909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.528990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.529071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.529153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.529232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.529308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.529388] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.529498] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.530766] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24897, lrc_seqno=24897, guc_id=0, flags=0x73 in no process [-1]
<7> [112.530769] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.530843] ------------[ cut here ]------------
<4> [112.530845] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.530846] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:157/6437
<4> [112.530921] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.530986] autofs4 [last unloaded: snd_hda_intel]
<4> [112.530990] CPU: 11 UID: 0 PID: 6437 Comm: kworker/u64:157 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.530993] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.530994] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.530996] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.531001] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.531072] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.531074] RSP: 0018:ffffc9000ed47ca0 EFLAGS: 00010246
<4> [112.531076] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.531077] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.531079] RBP: ffffc9000ed47db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.531080] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.531081] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.531082] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [112.531084] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.531085] CR2: 00005c658006a0b0 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.531087] PKRU: 55555554
<4> [112.531088] Call Trace:
<4> [112.531089] <TASK>
<4> [112.531093] ? lock_sync+0x100/0x100
<4> [112.531097] ? lock_release+0xd0/0x2b0
<4> [112.531103] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.531109] process_one_work+0x239/0x760
<4> [112.531115] worker_thread+0x200/0x3f0
<4> [112.531118] ? __pfx_worker_thread+0x10/0x10
<4> [112.531120] kthread+0x10d/0x150
<4> [112.531123] ? __pfx_kthread+0x10/0x10
<4> [112.531127] ret_from_fork+0x3d4/0x480
<4> [112.531129] ? __pfx_kthread+0x10/0x10
<4> [112.531132] ret_from_fork_asm+0x1a/0x30
<4> [112.531140] </TASK>
<4> [112.531141] irq event stamp: 322537
<4> [112.531142] hardirqs last enabled at (322543): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.531145] hardirqs last disabled at (322548): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.531147] softirqs last enabled at (321610): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.531150] softirqs last disabled at (321603): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.531152] ---[ end trace 0000000000000000 ]---
<6> [112.531154] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.531225] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.531233] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.531304] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.531407] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.531685] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.531890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.531995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.532091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.532181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.532268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.532354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.532439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.532536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.532619] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.532714] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.532830] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.533839] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.544850] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.545092] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.546108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.546185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.546256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.546329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.546399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.546468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.546551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.546626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.546701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.546775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.546848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.546922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.546996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.547070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.547143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.547217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.547314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.547398] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.547472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.547563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.547645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.547728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.547808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.547886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.547962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.548040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.548117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.548197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.548277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.548354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.548435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.548524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.548611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.548696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.548781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.548861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.548941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.549019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.549097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.549172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.549252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.549333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.549414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.549494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.549605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.549684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.549762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.549838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.549917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.549994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.550075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.550158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.550238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.550319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.550400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.550480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.550569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.550652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.550735] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.550853] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.550857] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24897, lrc_seqno=24897, guc_id=0, flags=0x73 in no process [-1]
<7> [112.550859] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.550918] ------------[ cut here ]------------
<4> [112.550919] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.550921] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:580/6860
<4> [112.550992] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.551057] autofs4 [last unloaded: snd_hda_intel]
<4> [112.551061] CPU: 10 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.551064] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.551065] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.551066] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.551071] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.551141] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.551143] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.551145] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.551146] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.551148] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.551149] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.551150] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.551151] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [112.551153] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.551154] CR2: 00007b19cb317000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.551156] PKRU: 55555554
<4> [112.551157] Call Trace:
<4> [112.551158] <TASK>
<4> [112.551162] ? lock_sync+0x100/0x100
<4> [112.551167] ? lock_release+0xd0/0x2b0
<4> [112.551172] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.551178] process_one_work+0x239/0x760
<4> [112.551184] worker_thread+0x200/0x3f0
<4> [112.551186] ? __pfx_worker_thread+0x10/0x10
<4> [112.551189] kthread+0x10d/0x150
<4> [112.551192] ? __pfx_kthread+0x10/0x10
<4> [112.551195] ret_from_fork+0x3d4/0x480
<4> [112.551197] ? __pfx_kthread+0x10/0x10
<4> [112.551201] ret_from_fork_asm+0x1a/0x30
<4> [112.551208] </TASK>
<4> [112.551209] irq event stamp: 234989
<4> [112.551210] hardirqs last enabled at (234995): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.551213] hardirqs last disabled at (235000): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.551215] softirqs last enabled at (233948): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.551218] softirqs last disabled at (233941): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.551220] ---[ end trace 0000000000000000 ]---
<5> [112.552525] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24952, lrc_seqno=24952, guc_id=0, flags=0x73 in no process [-1]
<7> [112.552533] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.552652] ------------[ cut here ]------------
<4> [112.552653] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.552654] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:580/6860
<4> [112.552738] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi
<7> [112.552764] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [112.552835] pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [112.552854] CPU: 11 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.552857] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.552858] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.552860] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.552867] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.552984] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.552987] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<7> [112.552914] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.552989]
<4> [112.552991] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.552993] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.552994] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.552996] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.552998] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.553000] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [112.553002] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.553004] CR2: 00005c658006a0b0 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.553005] PKRU: 55555554
<4> [112.553007] Call Trace:
<4> [112.553008] <TASK>
<4> [112.553013] ? lock_sync+0x100/0x100
<4> [112.553019] ? lock_release+0xd0/0x2b0
<4> [112.553028] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.553037] process_one_work+0x239/0x760
<4> [112.553048] worker_thread+0x200/0x3f0
<4> [112.553052] ? __pfx_worker_thread+0x10/0x10
<4> [112.553056] kthread+0x10d/0x150
<4> [112.553060] ? __pfx_kthread+0x10/0x10
<4> [112.553065] ret_from_fork+0x3d4/0x480
<4> [112.553068] ? __pfx_kthread+0x10/0x10
<4> [112.553072] ret_from_fork_asm+0x1a/0x30
<4> [112.553080] </TASK>
<4> [112.553081] irq event stamp: 236839
<4> [112.553082] hardirqs last enabled at (236845): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.553085] hardirqs last disabled at (236850): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.553087] softirqs last enabled at (233948): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.553089] softirqs last disabled at (233941): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.553092] ---[ end trace 0000000000000000 ]---
<6> [112.553093] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.553167] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.553173] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.553294] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.553498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.553632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.553725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.553816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.553905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.553993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.554079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.554167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.554249] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.554343] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.554460] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.555475] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.565508] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.565758] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.566726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.566806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.566883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.566960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.567036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.567112] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.567189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.567265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.567341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.567416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.567492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.567584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.567662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.567740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.567817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.567893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.567968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.568043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.568117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.568200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.568282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.568365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.568447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.568534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.568612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.568690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.568768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.568848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.568928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.569004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.569086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.569169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.569256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.569343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.569427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.569512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.569591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.569669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.569746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.569821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.569901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.569978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.570055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.570131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.570214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.570295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.570375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.570453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.570538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.570614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.570690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.570769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.570846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.570923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.570999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.571076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.571151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.571230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.571313] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.571425] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.571429] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24952, lrc_seqno=24952, guc_id=0, flags=0x73 in no process [-1]
<7> [112.571432] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.571490] ------------[ cut here ]------------
<4> [112.571491] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.571492] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:580/6860
<4> [112.571579] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.571653] autofs4 [last unloaded: snd_hda_intel]
<4> [112.571657] CPU: 11 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.571660] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.571661] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.571663] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.571669] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.571739] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.571741] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.571744] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.571745] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.571746] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.571748] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.571749] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.571751] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [112.571753] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.571754] CR2: 00005c658006a0b0 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.571756] PKRU: 55555554
<4> [112.571757] Call Trace:
<4> [112.571758] <TASK>
<4> [112.571762] ? lock_sync+0x100/0x100
<4> [112.571767] ? lock_release+0xd0/0x2b0
<4> [112.571773] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.571779] process_one_work+0x239/0x760
<4> [112.571785] worker_thread+0x200/0x3f0
<4> [112.571788] ? __pfx_worker_thread+0x10/0x10
<4> [112.571790] kthread+0x10d/0x150
<4> [112.571793] ? __pfx_kthread+0x10/0x10
<4> [112.571797] ret_from_fork+0x3d4/0x480
<4> [112.571799] ? __pfx_kthread+0x10/0x10
<4> [112.571803] ret_from_fork_asm+0x1a/0x30
<4> [112.571810] </TASK>
<4> [112.571812] irq event stamp: 240091
<4> [112.571813] hardirqs last enabled at (240097): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.571816] hardirqs last disabled at (240102): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.571818] softirqs last enabled at (238994): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.571820] softirqs last disabled at (238985): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.571823] ---[ end trace 0000000000000000 ]---
<6> [112.571824] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.571893] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.571899] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.573136] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.573243] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.573348] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.573567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.573662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.573755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.573844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.573932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.574017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.574105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.574194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.574280] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.574377] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.574494] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.575518] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.585514] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.585762] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.586775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.586857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.586934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.587012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.587089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.587165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.587240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.587316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.587390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.587465] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.587548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.587623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.587697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.587773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.587850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.587927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.588003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.588079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.588152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.588233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.588314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.588394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.588475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.588567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.588648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.588726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.588803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.588883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.588962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.589040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.589118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.589197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.589281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.589365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.589449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.589537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.589616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.589692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.589772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.589852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.589935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.590015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.590093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.590171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.590257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.590338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.590418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.590499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.590580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.590657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.590735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.590815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.590893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.590971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.591049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.591132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.591212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.591291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.591372] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.591483] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.591487] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=24952, lrc_seqno=24952, guc_id=0, flags=0x73 in no process [-1]
<7> [112.591490] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.592419] ------------[ cut here ]------------
<4> [112.592421] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.592422] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:155/6435
<4> [112.592503] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.592580] autofs4 [last unloaded: snd_hda_intel]
<4> [112.592585] CPU: 8 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.592588] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.592589] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.592591] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.592597] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.592668] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.592670] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.592673] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.592675] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.592676] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.592678] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.592679] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.592681] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.592683] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.592684] CR2: 00007b19cb31f000 CR3: 000000012654f004 CR4: 0000000000f72ef0
<4> [112.592686] PKRU: 55555554
<4> [112.592688] Call Trace:
<4> [112.592689] <TASK>
<4> [112.592693] ? lock_sync+0x100/0x100
<4> [112.592698] ? lock_release+0xd0/0x2b0
<4> [112.592704] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.592710] process_one_work+0x239/0x760
<4> [112.592716] worker_thread+0x200/0x3f0
<4> [112.592719] ? __pfx_worker_thread+0x10/0x10
<4> [112.592722] kthread+0x10d/0x150
<4> [112.592725] ? __pfx_kthread+0x10/0x10
<4> [112.592728] ret_from_fork+0x3d4/0x480
<4> [112.592730] ? __pfx_kthread+0x10/0x10
<4> [112.592734] ret_from_fork_asm+0x1a/0x30
<4> [112.592741] </TASK>
<4> [112.592742] irq event stamp: 223845
<4> [112.592743] hardirqs last enabled at (223851): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.592746] hardirqs last disabled at (223856): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.592749] softirqs last enabled at (223102): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.592751] softirqs last disabled at (223097): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.592754] ---[ end trace 0000000000000000 ]---
<5> [112.593124] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25007, lrc_seqno=25007, guc_id=0, flags=0x73 in no process [-1]
<7> [112.593131] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [112.593135] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [112.593242] ------------[ cut here ]------------
<4> [112.593243] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [112.593242] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.593246] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#13: kworker/u64:17/6297
<4> [112.593365] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.593457] autofs4 [last unloaded: snd_hda_intel]
<4> [112.593463] CPU: 13 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.593467] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.593469] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.593471] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.593479] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.593606] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.593609] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.593613] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.593616] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.593618] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.593620] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.593623] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.593625] FS: 0000000000000000(0000) GS:ffff8888db317000(0000) knlGS:0000000000000000
<4> [112.593628] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.593630] CR2: 00007b19cb324000 CR3: 000000012654f001 CR4: 0000000000f72ef0
<4> [112.593633] PKRU: 55555554
<4> [112.593635] Call Trace:
<4> [112.593637] <TASK>
<4> [112.593644] ? lock_sync+0x100/0x100
<4> [112.593651] ? lock_release+0xd0/0x2b0
<4> [112.593660] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.593670] process_one_work+0x239/0x760
<4> [112.593681] worker_thread+0x200/0x3f0
<4> [112.593685] ? __pfx_worker_thread+0x10/0x10
<4> [112.593689] kthread+0x10d/0x150
<4> [112.593693] ? __pfx_kthread+0x10/0x10
<4> [112.593699] ret_from_fork+0x3d4/0x480
<4> [112.593702] ? __pfx_kthread+0x10/0x10
<4> [112.593708] ret_from_fork_asm+0x1a/0x30
<4> [112.593720] </TASK>
<4> [112.593722] irq event stamp: 848527
<4> [112.593724] hardirqs last enabled at (848533): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.593728] hardirqs last disabled at (848538): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.593732] softirqs last enabled at (848366): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.593736] softirqs last disabled at (848355): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.593739] ---[ end trace 0000000000000000 ]---
<6> [112.593742] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.593859] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.593868] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.593964] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.594175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.594265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.594360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.594450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.594546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.594633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.594721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.594807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.594890] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.594984] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.595102] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.596110] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.606837] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.607089] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.608139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.608212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.608282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.608352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.608422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.608493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.608567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.608642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.608717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.608792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.608866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.608941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.609018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.609096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.609173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.609248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.609322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.609397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.609473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.609571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.609646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.609718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.609791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.609860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.609930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.610001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.610075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.610151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.610225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.610299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.610374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.610448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.610535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.610613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.610690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.610764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.610836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.610907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.610983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.611056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.611132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.611204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.611276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.611347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.611425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.611501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.611574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.611645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.611717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.611787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.611856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.611929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.612000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.612070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.612140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.612210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.612279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.612348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.612420] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.612528] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.613375] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25007, lrc_seqno=25007, guc_id=0, flags=0x73 in no process [-1]
<7> [112.613377] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.613434] ------------[ cut here ]------------
<4> [112.613435] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.613436] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:155/6435
<4> [112.613505] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.613565] autofs4 [last unloaded: snd_hda_intel]
<4> [112.613569] CPU: 8 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.613572] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.613573] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.613574] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.613579] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.613642] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.613644] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.613646] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.613648] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.613649] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.613651] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.613652] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.613653] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [112.613655] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.613657] CR2: 00007b19cb31f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.613658] PKRU: 55555554
<4> [112.613659] Call Trace:
<4> [112.613661] <TASK>
<4> [112.613664] ? lock_sync+0x100/0x100
<4> [112.613668] ? lock_release+0xd0/0x2b0
<4> [112.613673] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.613679] process_one_work+0x239/0x760
<4> [112.613685] worker_thread+0x200/0x3f0
<4> [112.613687] ? __pfx_worker_thread+0x10/0x10
<4> [112.613690] kthread+0x10d/0x150
<4> [112.613693] ? __pfx_kthread+0x10/0x10
<4> [112.613696] ret_from_fork+0x3d4/0x480
<4> [112.613698] ? __pfx_kthread+0x10/0x10
<4> [112.613701] ret_from_fork_asm+0x1a/0x30
<4> [112.613708] </TASK>
<4> [112.613709] irq event stamp: 228103
<4> [112.613710] hardirqs last enabled at (228109): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.613713] hardirqs last disabled at (228114): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.613716] softirqs last enabled at (227060): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.613718] softirqs last disabled at (227053): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.613720] ---[ end trace 0000000000000000 ]---
<6> [112.613722] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.613785] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.613793] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.613808] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.613903] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.614131] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.614386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.614572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.614714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.614849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.614983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.615115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.615249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.615385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.615523] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.615668] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.615836] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.617195] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.627496] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.627806] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.629025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.629147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.629266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.629388] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.629517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.629650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.629740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.629822] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.629900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.629977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.630052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.630128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.630203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.630279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.630353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.630428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.630511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.630596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.630672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.630753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.630836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.630919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.631003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.631082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.631161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.631239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.631316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.631396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.631474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.631584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.631672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.631751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.631836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.631922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.632010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.632095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.632177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.632257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.632335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.632414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.632507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.632598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.632679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.632757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.632839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.632921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.633003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.633083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.633163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.633241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.633318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.633396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.633475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.633574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.633665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.633746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.633823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.633900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.633981] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.634094] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.634098] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25007, lrc_seqno=25007, guc_id=0, flags=0x73 in no process [-1]
<7> [112.634101] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.634161] ------------[ cut here ]------------
<4> [112.634162] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.634163] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#11: kworker/u64:17/6297
<4> [112.634235] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.634301] autofs4 [last unloaded: snd_hda_intel]
<4> [112.634306] CPU: 11 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.634309] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.634311] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.634312] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.634318] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.634388] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.634390] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.634392] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.634394] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.634396] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.634397] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.634398] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.634400] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000
<4> [112.634402] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.634403] CR2: 00005c658006a0b0 CR3: 000000015b49d004 CR4: 0000000000f72ef0
<4> [112.634405] PKRU: 55555554
<4> [112.634407] Call Trace:
<4> [112.634408] <TASK>
<4> [112.634412] ? lock_sync+0x100/0x100
<4> [112.634417] ? lock_release+0xd0/0x2b0
<4> [112.634423] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.634428] process_one_work+0x239/0x760
<4> [112.634435] worker_thread+0x200/0x3f0
<4> [112.634438] ? __pfx_worker_thread+0x10/0x10
<4> [112.634441] kthread+0x10d/0x150
<4> [112.634444] ? __pfx_kthread+0x10/0x10
<4> [112.634448] ret_from_fork+0x3d4/0x480
<4> [112.634450] ? __pfx_kthread+0x10/0x10
<4> [112.634453] ret_from_fork_asm+0x1a/0x30
<4> [112.634460] </TASK>
<4> [112.634462] irq event stamp: 853053
<4> [112.634463] hardirqs last enabled at (853059): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.634466] hardirqs last disabled at (853064): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.634468] softirqs last enabled at (852088): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.634470] softirqs last disabled at (852081): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.634473] ---[ end trace 0000000000000000 ]---
<5> [112.635581] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25062, lrc_seqno=25062, guc_id=0, flags=0x73 in no process [-1]
<7> [112.635589] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [112.635595] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [112.635700] ------------[ cut here ]------------
<4> [112.635702] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [112.635704] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [112.635705] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:17/6297
<4> [112.635825] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.635917] autofs4 [last unloaded: snd_hda_intel]
<4> [112.635923] CPU: 12 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.635927] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.635929] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.635931] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.635939] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.636052] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.636055] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.636058] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.636060] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.636062] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.636064] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.636065] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.636067] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [112.636070] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.636072] CR2: 00007b19cb321000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.636074] PKRU: 55555554
<4> [112.636075] Call Trace:
<4> [112.636077] <TASK>
<4> [112.636084] ? lock_sync+0x100/0x100
<4> [112.636091] ? lock_release+0xd0/0x2b0
<4> [112.636099] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.636108] process_one_work+0x239/0x760
<4> [112.636118] worker_thread+0x200/0x3f0
<4> [112.636122] ? __pfx_worker_thread+0x10/0x10
<4> [112.636126] kthread+0x10d/0x150
<4> [112.636129] ? __pfx_kthread+0x10/0x10
<4> [112.636135] ret_from_fork+0x3d4/0x480
<4> [112.636137] ? __pfx_kthread+0x10/0x10
<4> [112.636142] ret_from_fork_asm+0x1a/0x30
<4> [112.636154] </TASK>
<4> [112.636156] irq event stamp: 854821
<4> [112.636157] hardirqs last enabled at (854827): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.636161] hardirqs last disabled at (854832): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.636164] softirqs last enabled at (852088): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.636168] softirqs last disabled at (852081): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.636171] ---[ end trace 0000000000000000 ]---
<6> [112.636174] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.636288] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.636297] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.636445] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.636711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.636847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.636986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.637120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.637252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.637383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.637524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.637661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.637792] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.637934] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.638100] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.639698] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.650521] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.650840] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.652023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.652148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.652269] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.652392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.652533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.652620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.652697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.652769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.652839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.652910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.652980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.653050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.653119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.653188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.653260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.653331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.653400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.653467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.653576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.653653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.653727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.653799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.653872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.653943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.654017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.654090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.654161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.654236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.654309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.654381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.654453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.654537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.654619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.654699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.654780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.654855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.654928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.654999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.655071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.655140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.655212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.655284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.655355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.655426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.655503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.655579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.655650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.655721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.655792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.655861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.655933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.656008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.656081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.656153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.656223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.656293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.656364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.656433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.656512] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.656625] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.656629] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25062, lrc_seqno=25062, guc_id=0, flags=0x73 in no process [-1]
<7> [112.656631] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.656684] ------------[ cut here ]------------
<4> [112.656686] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.656687] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:17/6297
<4> [112.656752] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.656821] autofs4 [last unloaded: snd_hda_intel]
<4> [112.656826] CPU: 6 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.656829] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.656830] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.656831] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.656836] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.656901] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.656902] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.656905] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.656907] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.656908] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.656909] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.656911] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.656912] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.656914] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.656915] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.656917] PKRU: 55555554
<4> [112.656918] Call Trace:
<4> [112.656919] <TASK>
<4> [112.656923] ? lock_sync+0x100/0x100
<4> [112.656928] ? lock_release+0xd0/0x2b0
<4> [112.656933] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.656938] process_one_work+0x239/0x760
<4> [112.656944] worker_thread+0x200/0x3f0
<4> [112.656946] ? __pfx_worker_thread+0x10/0x10
<4> [112.656949] kthread+0x10d/0x150
<4> [112.656951] ? __pfx_kthread+0x10/0x10
<4> [112.656955] ret_from_fork+0x3d4/0x480
<4> [112.656956] ? __pfx_kthread+0x10/0x10
<4> [112.656960] ret_from_fork_asm+0x1a/0x30
<4> [112.656966] </TASK>
<4> [112.656967] irq event stamp: 858145
<4> [112.656968] hardirqs last enabled at (858151): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.656971] hardirqs last disabled at (858156): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.656973] softirqs last enabled at (857104): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.656975] softirqs last disabled at (857097): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.656977] ---[ end trace 0000000000000000 ]---
<6> [112.656979] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.657043] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.657049] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.657231] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.657426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.657508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.657597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.657680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.657760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.657839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.657918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.657998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.658073] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.658160] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.658269] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.659263] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.669484] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.669731] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.670755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.670836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.670912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.670990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.671067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.671144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.671222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.671299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.671375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.671444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.671525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.671599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.671666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.671733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.671800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.671872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.671942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.672011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.672080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.672156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.672230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.672303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.672377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.672452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.672535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.672610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.672682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.672759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.672835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.672909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.672983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.673055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.673133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.673210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.673287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.673361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.673433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.673513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.673584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.673652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.673724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.673796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.673867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.673936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.674009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.674085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.674158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.674229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.674301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.674372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.674442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.674533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.674608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.674678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.674750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.674826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.674899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.674971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.675046] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.675150] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.675154] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25062, lrc_seqno=25062, guc_id=0, flags=0x73 in no process [-1]
<7> [112.675156] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.675210] ------------[ cut here ]------------
<4> [112.675211] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.675213] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:17/6297
<4> [112.675278] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.675337] autofs4 [last unloaded: snd_hda_intel]
<4> [112.675340] CPU: 6 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.675343] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.675344] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.675345] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.675350] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.675414] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.675415] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.675417] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.675418] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.675420] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.675421] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.675422] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.675423] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.675424] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.675426] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.675427] PKRU: 55555554
<4> [112.675428] Call Trace:
<4> [112.675429] <TASK>
<4> [112.675433] ? lock_sync+0x100/0x100
<4> [112.675437] ? lock_release+0xd0/0x2b0
<4> [112.675441] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.675447] process_one_work+0x239/0x760
<4> [112.675452] worker_thread+0x200/0x3f0
<4> [112.675455] ? __pfx_worker_thread+0x10/0x10
<4> [112.675457] kthread+0x10d/0x150
<4> [112.675460] ? __pfx_kthread+0x10/0x10
<4> [112.675463] ret_from_fork+0x3d4/0x480
<4> [112.675465] ? __pfx_kthread+0x10/0x10
<4> [112.675468] ret_from_fork_asm+0x1a/0x30
<4> [112.675478] </TASK>
<4> [112.675479] irq event stamp: 861287
<4> [112.675480] hardirqs last enabled at (861293): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.675483] hardirqs last disabled at (861298): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.675485] softirqs last enabled at (860396): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.675487] softirqs last disabled at (860339): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.675490] ---[ end trace 0000000000000000 ]---
<5> [112.675848] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25117, lrc_seqno=25117, guc_id=0, flags=0x73 in no process [-1]
<7> [112.675854] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.676217] ------------[ cut here ]------------
<4> [112.676220] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.676222] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#13: kworker/u64:580/6860
<4> [112.676337] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.676438] autofs4 [last unloaded: snd_hda_intel]
<4> [112.676445] CPU: 13 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.676449] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.676451] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.676454] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.676462] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.676585] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.676588] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.676592] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.676594] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.676597] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.676599] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.676601] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.676603] FS: 0000000000000000(0000) GS:ffff8888db317000(0000) knlGS:0000000000000000
<4> [112.676606] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.676608] CR2: 00007b19cb324000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.676610] PKRU: 55555554
<4> [112.676612] Call Trace:
<4> [112.676614] <TASK>
<4> [112.676621] ? lock_sync+0x100/0x100
<4> [112.676630] ? lock_release+0xd0/0x2b0
<4> [112.676639] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.676649] process_one_work+0x239/0x760
<4> [112.676660] worker_thread+0x200/0x3f0
<4> [112.676664] ? __pfx_worker_thread+0x10/0x10
<4> [112.676668] kthread+0x10d/0x150
<4> [112.676672] ? __pfx_kthread+0x10/0x10
<4> [112.676677] ret_from_fork+0x3d4/0x480
<4> [112.676680] ? __pfx_kthread+0x10/0x10
<4> [112.676685] ret_from_fork_asm+0x1a/0x30
<4> [112.676699] </TASK>
<4> [112.676700] irq event stamp: 242625
<4> [112.676702] hardirqs last enabled at (242631): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.676706] hardirqs last disabled at (242636): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.676709] softirqs last enabled at (241276): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [112.676713] softirqs last disabled at (241274): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [112.676716] ---[ end trace 0000000000000000 ]---
<6> [112.676782] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.676904] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.676913] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.676971] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.677078] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.677185] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.677388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.677482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.677576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.677665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.677751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.677835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.677919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.678006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.678088] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.678183] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.678302] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.679322] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.689481] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.689729] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.690746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.690828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.690907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.690986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.691064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.691140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.691215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.691291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.691370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.691447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.691538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.691615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.691691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.691766] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.691842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.691917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.691992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.692066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.692140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.692224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.692309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.692392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.692478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.692556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.692634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.692711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.692788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.692868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.692949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.693026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.693104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.693184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.693272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.693358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.693443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.693533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.693614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.693691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.693768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.693844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.693924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.694002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.694079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.694155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.694234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.694311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.694388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.694464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.694556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.694638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.694718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.694798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.694877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.694957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.695034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.695112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.695189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.695266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.695346] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.695457] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.695461] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25117, lrc_seqno=25117, guc_id=0, flags=0x73 in no process [-1]
<7> [112.695464] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.695544] ------------[ cut here ]------------
<4> [112.695545] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.695546] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:17/6297
<4> [112.695618] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.695682] autofs4 [last unloaded: snd_hda_intel]
<4> [112.695686] CPU: 6 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.695688] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.695690] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.695691] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.695696] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.695767] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.695768] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.695771] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.695772] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.695773] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.695775] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.695776] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.695777] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.695779] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.695780] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.695782] PKRU: 55555554
<4> [112.695783] Call Trace:
<4> [112.695784] <TASK>
<4> [112.695788] ? lock_sync+0x100/0x100
<4> [112.695792] ? lock_release+0xd0/0x2b0
<4> [112.695797] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.695803] process_one_work+0x239/0x760
<4> [112.695809] worker_thread+0x200/0x3f0
<4> [112.695812] ? __pfx_worker_thread+0x10/0x10
<4> [112.695814] kthread+0x10d/0x150
<4> [112.695817] ? __pfx_kthread+0x10/0x10
<4> [112.695821] ret_from_fork+0x3d4/0x480
<4> [112.695823] ? __pfx_kthread+0x10/0x10
<4> [112.695826] ret_from_fork_asm+0x1a/0x30
<4> [112.695833] </TASK>
<4> [112.695835] irq event stamp: 865653
<4> [112.695836] hardirqs last enabled at (865659): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.695838] hardirqs last disabled at (865664): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.695841] softirqs last enabled at (864892): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.695843] softirqs last disabled at (864885): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.695845] ---[ end trace 0000000000000000 ]---
<6> [112.695847] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.695919] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.695926] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.697166] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.697292] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.697417] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.697638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.697730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.697825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.697914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.698001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.698086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.698171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.698258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.698341] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.698438] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.698571] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.699598] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.709477] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.709723] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.710747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.710829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.710907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.710985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.711062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.711138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.711213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.711289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.711363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.711438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.711527] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.711605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.711681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.711756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.711831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.711905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.711980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.712054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.712128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.712209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.712291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.712372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.712454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.712548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.712626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.712703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.712786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.712866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.712949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.713031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.713113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.713194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.713279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.713364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.713448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.713539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.713619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.713697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.713774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.713855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.713937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.714017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.714095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.714171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.714252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.714328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.714406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.714490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.714569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.714648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.714728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.714810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.714888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.714966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.715044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.715122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.715199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.715275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.715354] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.715471] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.715476] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25117, lrc_seqno=25117, guc_id=0, flags=0x73 in no process [-1]
<7> [112.715478] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.715537] ------------[ cut here ]------------
<4> [112.715538] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.715540] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:580/6860
<4> [112.715612] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.715678] autofs4 [last unloaded: snd_hda_intel]
<4> [112.715682] CPU: 0 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.715685] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.715686] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.715687] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.715693] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.715763] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.715765] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.715767] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.715769] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.715770] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.715772] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.715773] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.715775] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.715777] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.715778] CR2: 00007b19cb31d000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.715780] PKRU: 55555554
<4> [112.715781] Call Trace:
<4> [112.715783] <TASK>
<4> [112.715787] ? lock_sync+0x100/0x100
<4> [112.715792] ? lock_release+0xd0/0x2b0
<4> [112.715798] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.715804] process_one_work+0x239/0x760
<4> [112.715811] worker_thread+0x200/0x3f0
<4> [112.715814] ? __pfx_worker_thread+0x10/0x10
<4> [112.715816] kthread+0x10d/0x150
<4> [112.715819] ? __pfx_kthread+0x10/0x10
<4> [112.715823] ret_from_fork+0x3d4/0x480
<4> [112.715826] ? __pfx_kthread+0x10/0x10
<4> [112.715829] ret_from_fork_asm+0x1a/0x30
<4> [112.715837] </TASK>
<4> [112.715838] irq event stamp: 245893
<4> [112.715840] hardirqs last enabled at (245899): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.715843] hardirqs last disabled at (245904): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.715846] softirqs last enabled at (244890): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.715848] softirqs last disabled at (244883): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.715851] ---[ end trace 0000000000000000 ]---
<5> [112.717069] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25172, lrc_seqno=25172, guc_id=0, flags=0x73 in no process [-1]
<7> [112.717072] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.717134] ------------[ cut here ]------------
<4> [112.717135] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.717136] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:580/6860
<4> [112.717206] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.717269] autofs4 [last unloaded: snd_hda_intel]
<4> [112.717273] CPU: 0 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.717276] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.717277] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.717278] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.717283] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.717351] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.717353] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.717355] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.717356] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.717357] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.717358] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.717359] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.717361] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.717362] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.717364] CR2: 00007b19cb31d000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.717365] PKRU: 55555554
<4> [112.717366] Call Trace:
<4> [112.717367] <TASK>
<4> [112.717371] ? lock_sync+0x100/0x100
<4> [112.717375] ? lock_release+0xd0/0x2b0
<4> [112.717380] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.717386] process_one_work+0x239/0x760
<4> [112.717392] worker_thread+0x200/0x3f0
<4> [112.717395] ? __pfx_worker_thread+0x10/0x10
<4> [112.717397] kthread+0x10d/0x150
<4> [112.717400] ? __pfx_kthread+0x10/0x10
<4> [112.717404] ret_from_fork+0x3d4/0x480
<4> [112.717405] ? __pfx_kthread+0x10/0x10
<4> [112.717409] ret_from_fork_asm+0x1a/0x30
<4> [112.717416] </TASK>
<4> [112.717417] irq event stamp: 247763
<4> [112.717418] hardirqs last enabled at (247769): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.717421] hardirqs last disabled at (247774): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.717423] softirqs last enabled at (244890): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.717426] softirqs last disabled at (244883): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.717428] ---[ end trace 0000000000000000 ]---
<6> [112.717429] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.717508] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.717515] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.717669] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.717774] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.717881] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.718088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.718176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.718268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.718357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.718444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.718568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.718659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.718748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.718830] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.718926] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.719045] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.720067] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.730472] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.730722] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.731761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.731840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.731918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.731995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.732074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.732151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.732228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.732305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.732382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.732460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.732551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.732627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.732702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.732778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.732852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.732926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.733000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.733075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.733148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.733230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.733312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.733394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.733481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.733561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.733639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.733716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.733794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.733874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.733954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.734033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.734111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.734190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.734274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.734357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.734440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.734528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.734612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.734694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.734773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.734851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.734932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.735010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.735088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.735164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.735243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.735320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.735397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.735477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.735555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.735633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.735709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.735788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.735866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.735942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.736018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.736096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.736171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.736247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.736325] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.736438] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.736442] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25172, lrc_seqno=25172, guc_id=0, flags=0x73 in no process [-1]
<7> [112.736444] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.736522] ------------[ cut here ]------------
<4> [112.736523] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.736524] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.736595] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.736660] autofs4 [last unloaded: snd_hda_intel]
<4> [112.736664] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.736666] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.736668] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.736669] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.736674] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.736745] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.736746] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.736748] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.736750] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.736751] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.736752] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.736753] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.736755] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.736756] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.736758] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.736759] PKRU: 55555554
<4> [112.736760] Call Trace:
<4> [112.736761] <TASK>
<4> [112.736765] ? lock_sync+0x100/0x100
<4> [112.736770] ? lock_release+0xd0/0x2b0
<4> [112.736775] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.736781] process_one_work+0x239/0x760
<4> [112.736787] worker_thread+0x200/0x3f0
<4> [112.736789] ? __pfx_worker_thread+0x10/0x10
<4> [112.736792] kthread+0x10d/0x150
<4> [112.736795] ? __pfx_kthread+0x10/0x10
<4> [112.736798] ret_from_fork+0x3d4/0x480
<4> [112.736800] ? __pfx_kthread+0x10/0x10
<4> [112.736804] ret_from_fork_asm+0x1a/0x30
<4> [112.736811] </TASK>
<4> [112.736812] irq event stamp: 236017
<4> [112.736813] hardirqs last enabled at (236023): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.736816] hardirqs last disabled at (236028): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.736818] softirqs last enabled at (235256): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.736821] softirqs last disabled at (235245): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.736823] ---[ end trace 0000000000000000 ]---
<6> [112.736825] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.736894] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.736900] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.738142] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.738268] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.738377] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.738586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.738680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.738776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.738867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.738956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.739042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.739128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.739217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.739300] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.739398] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.739523] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.740545] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.750471] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.750719] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.751682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.751763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.751842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.751920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.751997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.752074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.752149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.752225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.752302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.752381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.752458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.752552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.752628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.752704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.752779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.752853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.752928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.753003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.753077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.753159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.753241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.753320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.753402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.753486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.753565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.753643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.753721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.753802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.753887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.753969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.754050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.754132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.754218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.754302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.754386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.754470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.754550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.754631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.754713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.754792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.754873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.754952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.755029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.755106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.755190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.755272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.755353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.755430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.755517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.755593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.755670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.755750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.755828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.755906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.755983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.756061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.756138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.756214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.756293] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.756406] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.756411] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25172, lrc_seqno=25172, guc_id=0, flags=0x73 in no process [-1]
<7> [112.756413] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.756483] ------------[ cut here ]------------
<4> [112.756485] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.756486] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [112.756560] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.756626] autofs4 [last unloaded: snd_hda_intel]
<4> [112.756630] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.756633] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.756634] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.756636] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.756641] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.756714] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.756715] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.756718] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.756719] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.756721] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.756722] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.756723] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.756724] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [112.756726] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.756727] CR2: 00007b19cb326000 CR3: 000000012654f003 CR4: 0000000000f72ef0
<4> [112.756729] PKRU: 55555554
<4> [112.756730] Call Trace:
<4> [112.756731] <TASK>
<4> [112.756735] ? lock_sync+0x100/0x100
<4> [112.756740] ? lock_release+0xd0/0x2b0
<4> [112.756745] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.756751] process_one_work+0x239/0x760
<4> [112.756757] worker_thread+0x200/0x3f0
<4> [112.756760] ? __pfx_worker_thread+0x10/0x10
<4> [112.756762] kthread+0x10d/0x150
<4> [112.756765] ? __pfx_kthread+0x10/0x10
<4> [112.756769] ret_from_fork+0x3d4/0x480
<4> [112.756771] ? __pfx_kthread+0x10/0x10
<4> [112.756775] ret_from_fork_asm+0x1a/0x30
<4> [112.756782] </TASK>
<4> [112.756783] irq event stamp: 869375
<4> [112.756784] hardirqs last enabled at (869381): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.756787] hardirqs last disabled at (869386): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.756789] softirqs last enabled at (868614): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.756792] softirqs last disabled at (868591): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.756794] ---[ end trace 0000000000000000 ]---
<5> [112.758160] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25227, lrc_seqno=25227, guc_id=0, flags=0x73 in no process [-1]
<7> [112.758164] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.758227] ------------[ cut here ]------------
<4> [112.758229] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.758232] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [112.758303] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.758367] autofs4 [last unloaded: snd_hda_intel]
<4> [112.758370] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.758373] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.758374] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.758375] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.758381] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.758449] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.758451] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.758453] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.758454] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.758456] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.758457] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.758458] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.758465] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [112.758467] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.758468] CR2: 00007b19cb326000 CR3: 000000012654f003 CR4: 0000000000f72ef0
<4> [112.758470] PKRU: 55555554
<4> [112.758471] Call Trace:
<4> [112.758472] <TASK>
<4> [112.758476] ? lock_sync+0x100/0x100
<4> [112.758480] ? lock_release+0xd0/0x2b0
<4> [112.758486] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.758492] process_one_work+0x239/0x760
<4> [112.758498] worker_thread+0x200/0x3f0
<4> [112.758501] ? __pfx_worker_thread+0x10/0x10
<4> [112.758504] kthread+0x10d/0x150
<4> [112.758507] ? __pfx_kthread+0x10/0x10
<4> [112.758510] ret_from_fork+0x3d4/0x480
<4> [112.758512] ? __pfx_kthread+0x10/0x10
<4> [112.758516] ret_from_fork_asm+0x1a/0x30
<4> [112.758524] </TASK>
<4> [112.758525] irq event stamp: 871265
<4> [112.758526] hardirqs last enabled at (871271): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.758529] hardirqs last disabled at (871276): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.758532] softirqs last enabled at (871152): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.758535] softirqs last disabled at (871141): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.758537] ---[ end trace 0000000000000000 ]---
<6> [112.758539] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.758608] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.758614] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.758869] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.758972] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.759078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.759281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.759371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.759470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.759560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.759646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.759731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.759817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.759904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.759987] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.760084] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.760202] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.761228] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.771815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.772066] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.773087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.773160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.773230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.773301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.773371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.773443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.773530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.773607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.773684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.773760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.773834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.773909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.773982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.774056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.774130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.774204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.774280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.774358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.774433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.774523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.774606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.774685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.774767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.774844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.774927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.775007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.775087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.775168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.775250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.775330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.775411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.775498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.775584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.775669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.775754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.775836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.775917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.775996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.776079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.776159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.776243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.776324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.776406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.776492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.776576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.776657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.776735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.776813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.776893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.776971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.777047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.777126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.777203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.777281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.777358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.777440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.777534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.777614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.777698] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.777811] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.777815] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25227, lrc_seqno=25227, guc_id=0, flags=0x73 in no process [-1]
<7> [112.777818] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.777877] ------------[ cut here ]------------
<4> [112.777878] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.777880] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:17/6297
<4> [112.777952] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.778017] autofs4 [last unloaded: snd_hda_intel]
<4> [112.778021] CPU: 10 UID: 0 PID: 6297 Comm: kworker/u64:17 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.778024] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.778025] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.778027] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.778032] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.778101] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.778103] RSP: 0018:ffffc90007467ca0 EFLAGS: 00010246
<4> [112.778105] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.778106] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.778108] RBP: ffffc90007467db0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.778109] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.778110] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.778111] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000
<4> [112.778113] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.778114] CR2: 00007b19cb326000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.778116] PKRU: 55555554
<4> [112.778117] Call Trace:
<4> [112.778118] <TASK>
<4> [112.778122] ? lock_sync+0x100/0x100
<4> [112.778126] ? lock_release+0xd0/0x2b0
<4> [112.778132] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.778138] process_one_work+0x239/0x760
<4> [112.778144] worker_thread+0x200/0x3f0
<4> [112.778146] ? __pfx_worker_thread+0x10/0x10
<4> [112.778149] kthread+0x10d/0x150
<4> [112.778152] ? __pfx_kthread+0x10/0x10
<4> [112.778155] ret_from_fork+0x3d4/0x480
<4> [112.778157] ? __pfx_kthread+0x10/0x10
<4> [112.778161] ret_from_fork_asm+0x1a/0x30
<4> [112.778168] </TASK>
<4> [112.778169] irq event stamp: 874397
<4> [112.778170] hardirqs last enabled at (874403): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.778173] hardirqs last disabled at (874408): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.778175] softirqs last enabled at (873542): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.778178] softirqs last disabled at (873531): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.778180] ---[ end trace 0000000000000000 ]---
<6> [112.778182] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.778252] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.778258] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.779493] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.779599] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.779704] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.779907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.779995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.780090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.780182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.780270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.780356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.780442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.780543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.780627] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.780723] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.780841] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.781846] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.792568] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.792808] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.793827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.793906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.793984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.794061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.794139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.794217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.794293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.794370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.794446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.794540] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.794618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.794696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.794771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.794848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.794923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.794998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.795074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.795148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.795223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.795306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.795390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.795479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.795562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.795641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.795720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.795798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.795875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.795955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.796036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.796115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.796200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.796282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.796369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.796458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.796544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.796626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.796706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.796785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.796865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.796942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.797024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.797107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.797189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.797268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.797350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.797429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.797515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.797593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.797672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.797751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.797831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.797913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.797991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.798070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.798146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.798224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.798300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.798379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.798464] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.798577] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.798581] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25227, lrc_seqno=25227, guc_id=0, flags=0x73 in no process [-1]
<7> [112.798584] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.798642] ------------[ cut here ]------------
<4> [112.798643] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.798645] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.798718] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.798792] autofs4 [last unloaded: snd_hda_intel]
<4> [112.798796] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.798799] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.798801] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.798802] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.798808] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.798878] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.798879] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.798881] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.798883] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.798884] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.798886] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.798887] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.798888] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.798890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.798891] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.798893] PKRU: 55555554
<4> [112.798894] Call Trace:
<4> [112.798895] <TASK>
<4> [112.798899] ? lock_sync+0x100/0x100
<4> [112.798903] ? lock_release+0xd0/0x2b0
<4> [112.798909] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.798914] process_one_work+0x239/0x760
<4> [112.798921] worker_thread+0x200/0x3f0
<4> [112.798923] ? __pfx_worker_thread+0x10/0x10
<4> [112.798926] kthread+0x10d/0x150
<4> [112.798929] ? __pfx_kthread+0x10/0x10
<4> [112.798932] ret_from_fork+0x3d4/0x480
<4> [112.798934] ? __pfx_kthread+0x10/0x10
<4> [112.798938] ret_from_fork_asm+0x1a/0x30
<4> [112.798945] </TASK>
<4> [112.798946] irq event stamp: 239681
<4> [112.798947] hardirqs last enabled at (239687): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.798950] hardirqs last disabled at (239692): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.798952] softirqs last enabled at (238760): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.798955] softirqs last disabled at (238753): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.798957] ---[ end trace 0000000000000000 ]---
<5> [112.800314] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25282, lrc_seqno=25282, guc_id=0, flags=0x73 in no process [-1]
<7> [112.800317] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.800379] ------------[ cut here ]------------
<4> [112.800380] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.800381] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.800454] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.800519] autofs4 [last unloaded: snd_hda_intel]
<4> [112.800523] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.800525] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.800527] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.800528] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.800533] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.800602] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.800603] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.800605] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.800606] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.800608] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.800609] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.800610] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.800611] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.800613] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.800614] CR2: 00007b19cb31b000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [112.800615] PKRU: 55555554
<4> [112.800617] Call Trace:
<4> [112.800618] <TASK>
<4> [112.800621] ? lock_sync+0x100/0x100
<4> [112.800625] ? lock_release+0xd0/0x2b0
<4> [112.800631] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.800636] process_one_work+0x239/0x760
<4> [112.800643] worker_thread+0x200/0x3f0
<4> [112.800645] ? __pfx_worker_thread+0x10/0x10
<4> [112.800648] kthread+0x10d/0x150
<4> [112.800651] ? __pfx_kthread+0x10/0x10
<4> [112.800655] ret_from_fork+0x3d4/0x480
<4> [112.800657] ? __pfx_kthread+0x10/0x10
<4> [112.800661] ret_from_fork_asm+0x1a/0x30
<4> [112.800668] </TASK>
<4> [112.800670] irq event stamp: 241553
<4> [112.800671] hardirqs last enabled at (241559): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.800673] hardirqs last disabled at (241564): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.800676] softirqs last enabled at (240812): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.800679] softirqs last disabled at (240807): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.800681] ---[ end trace 0000000000000000 ]---
<6> [112.800683] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.800752] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.800757] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.800990] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.801095] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.801204] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.801409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.801511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.801605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.801694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.801782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.801869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.801956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.802045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.802130] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.802227] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.802345] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.803354] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.814077] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.814327] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.815489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.815623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.815748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.815872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.815994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.816116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.816237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.816358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.816495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.816618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.816739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.816861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.816983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.817105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.817226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.817347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.817474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.817598] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.817719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.817847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.817974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.818099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.818226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.818351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.818480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.818605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.818730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.818858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.818986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.819114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.819241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.819368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.819521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.819612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.819702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.819790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.819874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.819952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.820031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.820106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.820187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.820265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.820341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.820417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.820509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.820587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.820664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.820744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.820826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.820905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.820983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.821063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.821140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.821218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.821295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.821372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.821451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.821527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.821606] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.821718] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.821722] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25282, lrc_seqno=25282, guc_id=0, flags=0x73 in no process [-1]
<7> [112.821725] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.821784] ------------[ cut here ]------------
<4> [112.821786] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.821787] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.821860] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.821925] autofs4 [last unloaded: snd_hda_intel]
<4> [112.821929] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.821932] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.821933] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.821935] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.821940] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.822011] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.822013] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.822015] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.822016] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.822018] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.822019] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.822020] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.822021] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.822023] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.822024] CR2: 00007b19cb31b000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [112.822026] PKRU: 55555554
<4> [112.822027] Call Trace:
<4> [112.822028] <TASK>
<4> [112.822032] ? lock_sync+0x100/0x100
<4> [112.822036] ? lock_release+0xd0/0x2b0
<4> [112.822042] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.822047] process_one_work+0x239/0x760
<4> [112.822054] worker_thread+0x200/0x3f0
<4> [112.822057] ? __pfx_worker_thread+0x10/0x10
<4> [112.822059] kthread+0x10d/0x150
<4> [112.822062] ? __pfx_kthread+0x10/0x10
<4> [112.822065] ret_from_fork+0x3d4/0x480
<4> [112.822067] ? __pfx_kthread+0x10/0x10
<4> [112.822071] ret_from_fork_asm+0x1a/0x30
<4> [112.822078] </TASK>
<4> [112.822079] irq event stamp: 244695
<4> [112.822080] hardirqs last enabled at (244701): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.822083] hardirqs last disabled at (244706): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.822085] softirqs last enabled at (243840): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.822088] softirqs last disabled at (243833): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.822090] ---[ end trace 0000000000000000 ]---
<6> [112.822092] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.822162] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.822168] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.823415] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.823552] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.823665] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.823883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.823976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.824072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.824162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.824252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.824344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.824434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.824544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.824630] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.824726] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.824843] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.825852] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.836451] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.836699] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.837715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.837796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.837873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.837950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.838028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.838103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.838178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.838253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.838328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.838403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.838492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.838568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.838646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.838722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.838797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.838871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.838948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.839026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.839102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.839184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.839266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.839346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.839426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.839521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.839599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.839681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.839763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.839846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.839934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.840015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.840096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.840177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.840263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.840349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.840434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.840537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.840623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.840703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.840784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.840863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.840944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.841024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.841102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.841179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.841261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.841338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.841415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.841505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.841583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.841659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.841735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.841815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.841892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.841971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.842048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.842126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.842203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.842284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.842369] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.842490] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.842494] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25282, lrc_seqno=25282, guc_id=0, flags=0x73 in no process [-1]
<7> [112.842497] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.842556] ------------[ cut here ]------------
<4> [112.842557] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.842559] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:580/6860
<4> [112.842631] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.842698] autofs4 [last unloaded: snd_hda_intel]
<4> [112.842703] CPU: 0 UID: 0 PID: 6860 Comm: kworker/u64:580 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.842706] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.842707] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.842709] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.842714] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.842785] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.842787] RSP: 0018:ffffc9000fa5fca0 EFLAGS: 00010246
<4> [112.842789] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.842791] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.842792] RBP: ffffc9000fa5fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.842794] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.842795] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.842797] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000
<4> [112.842799] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.842800] CR2: 00007b19cb31d000 CR3: 000000000344c001 CR4: 0000000000f72ef0
<4> [112.842802] PKRU: 55555554
<4> [112.842803] Call Trace:
<4> [112.842805] <TASK>
<4> [112.842809] ? lock_sync+0x100/0x100
<4> [112.842814] ? lock_release+0xd0/0x2b0
<4> [112.842819] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.842825] process_one_work+0x239/0x760
<4> [112.842832] worker_thread+0x200/0x3f0
<4> [112.842835] ? __pfx_worker_thread+0x10/0x10
<4> [112.842837] kthread+0x10d/0x150
<4> [112.842841] ? __pfx_kthread+0x10/0x10
<4> [112.842844] ret_from_fork+0x3d4/0x480
<4> [112.842847] ? __pfx_kthread+0x10/0x10
<4> [112.842850] ret_from_fork_asm+0x1a/0x30
<4> [112.842858] </TASK>
<4> [112.842859] irq event stamp: 255287
<4> [112.842860] hardirqs last enabled at (255293): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.842863] hardirqs last disabled at (255298): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.842866] softirqs last enabled at (254284): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.842868] softirqs last disabled at (254277): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.842871] ---[ end trace 0000000000000000 ]---
<5> [112.843467] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25337, lrc_seqno=25337, guc_id=0, flags=0x73 in no process [-1]
<7> [112.843473] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.843575] ------------[ cut here ]------------
<4> [112.843577] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.843579] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.843654] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.843731] autofs4 [last unloaded: snd_hda_intel]
<4> [112.843736] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.843739] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.843740] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.843742] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.843751] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.843824] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.843826] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.843828] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.843830] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.843831] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.843833] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.843834] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.843835] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.843837] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.843838] CR2: 00007b19cb31b000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [112.843840] PKRU: 55555554
<4> [112.843841] Call Trace:
<4> [112.843842] <TASK>
<4> [112.843847] ? lock_sync+0x100/0x100
<4> [112.843852] ? lock_release+0xd0/0x2b0
<4> [112.843858] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.843864] process_one_work+0x239/0x760
<4> [112.843871] worker_thread+0x200/0x3f0
<4> [112.843875] ? __pfx_worker_thread+0x10/0x10
<4> [112.843877] kthread+0x10d/0x150
<4> [112.843880] ? __pfx_kthread+0x10/0x10
<4> [112.843884] ret_from_fork+0x3d4/0x480
<4> [112.843886] ? __pfx_kthread+0x10/0x10
<4> [112.843890] ret_from_fork_asm+0x1a/0x30
<4> [112.843897] </TASK>
<4> [112.843899] irq event stamp: 249029
<4> [112.843900] hardirqs last enabled at (249035): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.843903] hardirqs last disabled at (249040): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.843905] softirqs last enabled at (248082): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.843908] softirqs last disabled at (248075): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.843911] ---[ end trace 0000000000000000 ]---
<6> [112.843914] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [112.843999] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [112.844008] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [112.844356] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [112.844465] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [112.844571] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [112.844773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [112.844863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [112.844957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [112.845046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [112.845138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [112.845228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [112.845316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [112.845405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [112.845497] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [112.845594] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [112.845712] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [112.846734] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [112.857447] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [112.857697] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [112.858707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [112.858787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [112.858862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [112.858939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [112.859016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [112.859093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [112.859170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [112.859249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [112.859325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [112.859402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [112.859487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [112.859562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [112.859636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [112.859711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [112.859784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [112.859858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [112.859932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [112.860007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [112.860081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [112.860163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [112.860246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [112.860325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [112.860405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [112.860493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [112.860569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [112.860647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [112.860722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [112.860801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [112.860880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [112.860959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [112.861043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [112.861125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [112.861212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [112.861297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [112.861383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [112.861470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [112.861550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [112.861628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [112.861706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [112.861781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [112.861860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [112.861938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [112.862019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [112.862097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [112.862180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [112.862258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [112.862337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [112.862412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [112.862506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [112.862583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [112.862661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [112.862742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [112.862819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [112.862898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [112.862975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [112.863053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [112.863130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [112.863207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [112.863286] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [112.863399] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [112.863403] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=25337, lrc_seqno=25337, guc_id=0, flags=0x73 in no process [-1]
<7> [112.863406] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [112.864319] ------------[ cut here ]------------
<4> [112.864320] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [112.864322] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:155/6435
<4> [112.864397] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic asus_nb_wmi cmdlinepart coretemp asus_wmi spi_nor sparse_keymap mei_pxp mei_hdcp platform_profile mtd binfmt_misc wmi_bmof kvm_intel usbhid hid kvm video irqbypass ghash_clmulni_intel aesni_intel snd_intel_dspcfg snd_hda_codec rapl snd_hda_core intel_cstate r8169 snd_hwdep snd_pcm realtek snd_timer intel_pmc_core i2c_i801 idma64 mei_me snd i2c_mux spi_intel_pci i2c_smbus spi_intel soundcore pmt_telemetry mei pmt_discovery nls_iso8859_1 wmi pmt_class intel_pmc_ssram_telemetry intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
<4> [112.864468] autofs4 [last unloaded: snd_hda_intel]
<4> [112.864473] CPU: 6 UID: 0 PID: 6435 Comm: kworker/u64:155 Tainted: G S U W 7.0.0-lgci-xe-xe-4921-898b5aa235c5b269d-debug+ #1 PREEMPT(lazy)
<4> [112.864475] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [112.864477] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [112.864478] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [112.864484] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [112.864555] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 de 5e e1 48 89 c6 48 8d 3d 6c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [112.864556] RSP: 0018:ffffc9000d54fca0 EFLAGS: 00010246
<4> [112.864558] RAX: ffffffffa12046cb RBX: 0000000000000000 RCX: 0000000000000000
<4> [112.864560] RDX: ffff888101787710 RSI: ffffffffa12046cb RDI: ffffffffa1003ef0
<4> [112.864561] RBP: ffffc9000d54fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [112.864562] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [112.864563] R13: ffff888101787710 R14: ffff88816b33b818 R15: 00000000ffffffc2
<4> [112.864565] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [112.864566] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [112.864568] CR2: 00007b19cb31b000 CR3: 000000012654f005 CR4: 0000000000f72ef0
<4> [112.864569] PKRU: 55555554
<4> [112.864570] Call Trace:
<4> [112.864572] <TASK>
<4> [112.864575] ? lock_sync+0x100/0x100
<4> [112.864580] ? lock_release+0xd0/0x2b0
<4> [112.864585] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [112.864591] process_one_work+0x239/0x760
<4> [112.864597] worker_thread+0x200/0x3f0
<4> [112.864600] ? __pfx_worker_thread+0x10/0x10
<4> [112.864603] kthread+0x10d/0x150
<4> [112.864606] ? __pfx_kthread+0x10/0x10
<4> [112.864610] ret_from_fork+0x3d4/0x480
<4> [112.864612] ? __pfx_kthread+0x10/0x10
<4> [112.864615] ret_from_fork_asm+0x1a/0x30
<4> [112.864622] </TASK>
<4> [112.864624] irq event stamp: 252183
<4> [112.864625] hardirqs last enabled at (252189): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [112.864628] hardirqs last disabled at (252194): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [112.864631] softirqs last enabled at (251416): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.864633] softirqs last disabled at (251409): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [112.864636] ---[ end trace 0000000000000000 ]---
<6> [112.864638] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
Created at 2026-04-21 19:45:47